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ABSTRACT 


The purpose of this thesis is to analyze protests 
of contract awards brought before the Comptroller General, 
General Accounting Office from January 1998 through 
December 1999 as a means to identify areas of possible 
improvement among Federal contracting agencies. Specific 
emphasis is on the underlying causes of protest 
sustainment. This thesis distills eight sustaining 
elements from the GAO findings. Finally, this thesis 
offers recommendations to Federal contracting agencies in 
an effort to help mitigate the risk of a sustainable 
contract award protest. 
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I. INTRODUCTION 


A. DISCUSSION 

Since the release of the Federal Acquisition 
Regulation (FAR) Part 15 rewrite, Federal contracting 
officers have been encouraged to obtain the best value to 
the Government by conducting trade-offs when making source 
selections. If, in the contracting officer's opinion, it 
is in the best interests of the Government to award a 
contract to other than the lowest-priced, technically 
acceptable offeror or to the highest technically rated 
offeror, then he or she may make trade-offs. One 
stipulation for making trade-offs is that they be properly 
documented and supported by a reasoned explanation. 

Even assuming that a contracting officer follows this 
procedure, there is nothing to stop an unsuccessful offeror 
from filing a protest of the award with the General 
Accounting Office (GAO). Even though GAO's decisions are 
not binding on Federal agencies, they are highly 
influential and signal Congress's intent to ensure full and 
open competition in Government procurement. Protests 
arising from contract awards have serious potential for 
both adding costs to an acquisition and delaying delivery 
to the end user. An analysis of GAO's decisions should 
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reveal ways to mitigate the risk of sustainable contract 
award protests. 

B. OBJECTIVE 

This study will benefit both current and prospective 
Federal Government contracting officers to execute their 
duties. For the price of a postage stamp, disgruntled 
would-be contractors can bring the acquisition process to a 
grinding halt. This can result in increased costs to the 
Government and significant delays in delivering items to 
the end user. 

Contracting officers will be better prepared to 
mitigate the risk of sustainable contract award protests if 
they are aware of: 1) the historically common reasons cited 
in contract protests; and 2) which categories of protests 
of awards using the best value trade-off process have high 
sustainment rates. 

C. RESEARCH QUESTIONS 

1. Primary Research Question 

To what extent will an analysis of General Accounting 
Office decisions of protests of contracts awarded using the 
best value trade-off process provide insight into the 
nature of Government contract protests and suggest ways to 
reduce such protests in the future? 
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2. 


Secondary Research Questions 


• What will an analysis of Government contract 
protests from January 1998 through December 1999 
suggest about the nature of Government contract 
protests? 

• Do sustained protests of contracts awarded using the 
best value trade-off process fall into distinct 
categories? 

• What will an analysis of the research results 
suggest about weaknesses in Government contracting 
norms and execution practices? 

• How can the results of this analysis be used to 
reduce the risk of protest? 

• How can the results of this analysis be used to 
reduce the risk of a sustainable protest? 

D. SCOPE 

This thesis will include: 1) a review of all decisions 
rendered during calendar years 1998 and 1999 by the 
Comptroller General, General Accounting Office, in response 
to protests of contract award; and 2) an in-depth analysis 
of protests of contract awards using the best value trade¬ 
off source selection process. The thesis will conclude 
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with recommendations to contracting officers for mitigating 
the risk of sustainable protests of contract award. 

E. METHODOLOGY 

This study was conducted using the following methods: 

• Research using LEXIS/NEXIS to identify and acquire 
applicable cases. 

• Review and analysis of each protest brought before 
the Comptroller General and decided during calendar 
years 1998 and 1999. Estimated total: 558 cases. 

• Thorough examination of all protests of contracts 
awarded using the best value trade-off source 
selection process. 

• Categorization and analysis of sustained protests in 
order to generate recommendations and conclusions 
regarding sustained protests. 

F. DEFINITIONS 

The following definitions will allow the reader to 
better understand the analysis in the following chapters. 

In addition, Appendix A provides a listing of selected 
acronyms used throughout this thesis. 

1. Acquisition 

"Acquisition” means the acquiring by contract with 
appropriated funds of supplies or services (including 
construction) by and for the use of the Federal Government 
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through purchase or lease, whether the supplies or services 
are already in existence or must be created, developed, 
demonstrated, and evaluated. Acquisition begins when 
agency needs are established and includes the following: 
description of requirements to satisfy agency needs; 
solicitation and selection of sources;.award of contracts; 
contract financing; contract performance; contract 
administration; and those technical and management 
functions directly related to the process of fulfilling 
agency needs by contract. [Ref. 1: p.P-3] 

2. Best. Value 

"Best Value" means the expected outcome of an 
acquisition that, in the Government's estimation, provides 
the greatest overall benefit in response to the requirement 
for supplies or services. To obtain the greatest overall 
benefit in response to the requirement, one must consider 
the impact of every decision and action on price, quality, 
timeliness, risks, competition, socio-economic 
requirements, and compliance with ethical standards. 

[Ref 1: p.P-3] 

3. Contract 

A contract is a mutually binding legal relationship 
obligating the seller to furnish the supplies or services 
(including construction) and the buyer to pay for them. It 
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includes all types of coiranitments that obligate the 
Government to an expenditure of appropriated funds; these 
commitments are, except as otherwise authorized, in 
writing. In addition to bilateral instruments, contracts 
include (but are not limited to) awards and notices of 
awards; job orders or task letters issued under basic 
ordering agreements; letter contracts; orders, such as 
purchase orders, under which the contract becomes effective 
by written acceptance or performance; and bilateral 
contract modifications. Contracts do not include grants 
and cooperative agreements covered by 31 U.S.C. §6301. 

[Ref. 2:2.101] 

4. In'beres'bed Party 

An interested party is an actual or prospective bidder 
or offeror with a direct economic interest in the 
procurement. In challenges of the evaluation of proposals 
and the award of contracts, this generally means an offeror 
that would be in line for award if the protest were 
sustained. [Ref. 3: p.9] 

5. Offer 

"Offer" means a response to a solicitation that, if 
accepted, would bind the offeror to perform the resultant 
contract. Responses to invitations for bids (sealed 
bidding) are offers called "bids” or "sealed bids"; 
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responses to requests for proposals (negotiation) are 
offers called "proposals"; responses to requests for 
quotations (negotiation) are not offers and are called 
"quotes." [Ref. 2: 2.1] 

6. Protest 

Protest means a written objection by an interested 
party to any of the following: 

• A solicitation or other request by an agency for 
offers for a contract for the procurement of 
property or services. 

• The cancellation of the solicitation or other 
request. 

• An award or proposed award of the contract. 

• Termination or cancellation of an award of the 
contract if the written objection contains an 
allegation that the termination or cancellation is 
based in whole or in part on improprieties 
concerning the award of the contract. 

[Ref. 2: 33.101] 

7. Source Selection Authority (SSA) 

The SSA is the person responsible for making the 
source selection decision. While the SSA may use reports 
and analyses prepared by others, the source selection 
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decision must represent the SSA's independent judgment. 
[Ref. 2:15.308] 

8. Source Selection Evaluation Board (SSEB) 

The team that actually reviews the contractor's 
proposals may be known as the "Source Selection Evaluation 
Board (SSEB)," "Source Evaluation Board (SEB)," "Source 
Evaluation Team (SET)," or another similar name. These 
experts may be further divided into sub-teams to evaluate 
different aspects of each contractor's proposal (e.g., 
cost/price, technical, and past performance). [Ref. 4: 8.2] 

9. Trade-off Process 

In negotiated procurements, the most common approach 
to achieve a best-value selection is a trade-off process in 
which dollars are traded for incremental value, and cost is 
only one factor, not the controlling factor. A trade-off 
process is appropriate when it is in the Government's best 
interest to consider award to other than the lowest-priced 
offeror or other than the highest technically rated 
offeror. [Ref. 2: 15,101-1] 

6. ORGANIZATION 

Chapter I of this thesis presented the research 
questions used to gain insight into the nature of protests 
of contracts awarded using the best value trade-off 
process. Additionally, this chapter discussed the scope of 
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the thesis, the researcher's methodology, and the relevant 
terminology used throughout this study. 

Chapter II is a review of the trade-off process used 
to obtain the best value for Federal Government contracts, 
including the origin of trade-offs, the development of the 
trade-off process, and the integrity of the process. The 
chapter also provides a brief overview of the protest 
process. 

Chapter III is a quantitative analysis of sustained 
protests of contracts awarded using the best value trade¬ 
off process for calendar years 1998 and 1999, including a 
description of the methodology used to obtain the data. 

Chapter IV will present a qualitative analysis of 
sustained protests of contracts awarded using the best 
value trade-off process for calendar years 1998 and 1999. 
This chapter also categorizes the recurring reasons for 
sustained protests and identifies common weaknesses that 
contributed to these protests being sustained. 

Chapter V presents the conclusions and recommendations 
generated from this research, as well as suggested areas 
for further research. 
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II. BACKGROUND 


A. INTRODUCTION 

This chapter will present an overview of the trade-off 
process used to obtain the best value for Federal 
Government contracts. The purpose of this chapter is to 
provide the reader with a basic understanding of the trade¬ 
off process. For a more detailed discussion, see Appendix 

B, which contains excerpts from "AMC pamphlet 715-3: 
Contracting for Best Value; A Best Practices Guide to 
Source Selection." Specifically, this chapter will stress 
the importance of maintaining integrity in the trade-off 
process. Finally, the chapter will provide a brief 
overview of the protest process. 

1. Review of the Trade-off Process in Contracting 
Acquisition professionals, over use the term "Best- 
value" as it is commonly encountered. According to one 
source, "Best-value" means buying on other than a price- 
only basis, the usual Government practices."[Ref. 5: p.3-1] 
Every acquisition brokered on behalf of the Federal 
Government must be awarded "to the responsible source whose 
proposal is most advantageous to the United States, 
considering only cost or price and the other factors 
included in the solicitation." 
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[Ref. 3: U.S.C. §2305(b) (4) (c)]. By definition, a contract 
award must be determined to be in the Government's best 
interests. In other words, a best-value award is one that 
is made to the offeror proposing the lowest price or the 
highest technically rated offer, or some combination of the 
two as long as the award is determined to be the most 
advantageous to the United States. For this reason, the 
researcher has chosen to use the term "trade-off process" 
to identify contracts awarded using the trade-off 
decisional rule. 

A trade-off process is appropriate when it is 
essential to evaluate and compare factors other than cost 
or price in order to select the proposal that is in the 
Government's best interest. Trade-offs sometimes mean that 
awards are not made to those who offer the lowest price or 
the highest technically rated offeror. [Ref. 2: 15.101(a)] 

The following are instances in which the trade-off 
process may be particularly appropriate: 

• Government requirements are complex or difficult to 
define. 

• Services are not clearly defined, or highly skilled 
personnel are required. 
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• The Government is willing to pay extra for 
capability, skills, or reduced risk if the added benefits 
are worth the price premium. [App. B:p.120-121] 

A best-value award using the trade-off process then 
falls along the continuum that includes: 1) lowest priced, 
technically acceptable (LPTA) and 2) highest technically 
rated offer. A frequently cited range of alternatives—the 
best-value continuum—is often used to describe this trade¬ 
off of alternatives along the spectrum of options noted 
above. [Ref. 2: 15.101] 

For the purposes of this thesis, all contract awards 
discussed will be those evaluated solely using the trade¬ 
off process. 

2. Integrity of the Trade-off Process 

One of the fundamental concerns of Government 
contracting officers is maintaining the public's trust—the 
trust of prospective contractors as well as of the general 
public. Maintaining the integrity of the procurement 
process is not only an ethical issue, but it is also a 
legal one since there are both civil and criminal penalties 
for violations. [App. B: p.ll6] 

One way contracting officers in negotiated 
procurements maintain this trust is to award contracts 
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solely in accordance with the stated evaluation criteria 
specified in each solicitation. [Ref: 2:1.102] 

Specifically, all factors and significant sub-factors 
that will affect contract award must be stated clearly in 
the solicitation, along with their relative importance. 
Negotiated procurements can be complex, including first- 
level evaluators, a source selection evaluation board 
(SSEB), a source selection advisory council (SSAC) and, 
ultimately, the source selection authority (SSA). The 
illustration on page six of Appendix B provides an example 
of a complex source selection team. 

Alternatively, negotiated procurements can be less 
complex and include only first-level evaluators and the 
SSA. The diagram on page seven of Appendix B illustrates a 
typical non-complex source selection team. 

[Ref. 2:15.304(d)] 

By evaluating each proposal strictly against the 
factors and sub-factors stated in the solicitation, and not 
by comparison to the proposals of other offerors, the 
first-level evaluators do their part to maintain the 
integrity of the source selection process. Depending on 
the complexity of the procurement, the first-level 
evaluators provide the results of their evaluation either 
to the SSAC or directly to the SSA. For procurements that 
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require a complex source selection team, the evaluators 
forward their evaluation to the SSAC, which will compare 
the various offers and make an award recommendation to the 
SSA. In less-complex procurements, the SSA receives the 
evaluations directly from the first-level evaluators and 
performs the tasks of both comparing offers and selecting 
the offer(s) for award. By adhering to this policy, the 
source selection team helps ensure that the public's trust 
is maintained. 

The SSA then—as the final decision authority—must 
make an award decision based on a comparative assessment of 
proposals against all source selection criteria. The SSA 
must consider all available information, including reports 
and analyses prepared by others, in making this decision. 
This award decision must represent the SSA's independent 
judgment and must be adequately documented. The 
documentation should include the rationale for any business 
judgments and trade-offs made by the SSA, as well as the 
benefits associated with any price premiums accepted. 

[Ref. 2:15.308] 

B. PROTEST PROCESS 

Any interested party in the acquisition process, 
either because of frustrated feelings or suspicions of 
unfair treatment, may file a formal protest. Such protests 
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may cost the Government a lot of time and administrative 
effort, as well as the involvement of the entire 
acquisition team. Additionally, with limited exceptions, a 
protest will suspend performance of the contract pending a 
decision by the agency or the Comptroller General. It is, 
therefore, in everyone's best interests to avoid any 
actions—^perceived or actual—that could prompt a 
prospective contractor to file a formal protest. 

Contractors have several avenues for attempting to 
resolve contract award controversies. This thesis will 
consider only those protests filed with the Comptroller 
General for calendar years 1998 and 1999. The following 
paragraphs will outline: 1) how a formal protest is filed; 
2) an agency's action upon receiving a protest; 3) avenues 
open to the protestor upon receiving the agency's response; 
and 4) the protest hearing process and the General 
Accounting Office's (GAO) decision. Table 1 below presents 
a snapshot of the intricacies involved when a contract 
award is contested. 
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TABLE 1. 

CONTRACT AWARD CONTROVERSIES 


1 CONTRACT AWARD CONTROVERSIES I 

Forum 

Procuring 

Agency 

Court of 
Federal Claims 
(COFC) 

District Courts 

GAO 

Who May 
Protest 

Interested 

parties 

Persons with 
implied 
contract for 
good faith 
consideration 
of offer and 
interested 
parties 

Persons 
adversely 
affected or 
aggrieved by 
agency action 
and interested 
parties 

Interested 

parties 

Jurisdiction 

FAR 33.103 

28 U.S.C. 
§1491(a)(l) and 
(b) {2} 

5 U.S.C. §702, 

28 U.S.C. 

§1346(2) and 28 
U.S.C. 

§1491(b)(2) 

31 U.S.C. 

§3551, with 
deference to 
actions in 
other forums 

Time for 
Protest 

Pre- and post 
award 

Pre- and post 
award 

Pre- and post 
award 

Pre- and post 
award 

Time of 
Decision 

No designated 
time 

No designated 
time 

No designated 
time 

100 calendar 
days (65 
calendar days - 
express option) 

Procedures 

Various agency 
procedures; 
Informal in 
nature 

Evidentiary 

trial, 

including 

discovery 

Evidentiary 
trial, including 
discovery 

Submission of 
written reports 
and optional 
hearing 

Burden of 

Proof 

Probably same 
as GAO 

; Clear and 
convincing 
evidence of 
violation 

Preponderance of 
; evidence 

Presumption of 
correctness of 
agency action 

Standards of 
Review 

Violation of 
statute or 
regulation 

No rational 
basis or clear 
violation of 
statute or 
regulation 

No rational 
basis or clear 
violation of 
statute or 
regulation 

Violation of 
statute or 
regulation 

Interim 

Remedies 

: Agency must 
delay award or 
suspend 
' performance 
unless CICA 
override 

Discretionary 

temporary 

restraining 

order or 

preliminary 

injunction 

Discretionary 

temporary 

restraining 

order or 

preliminary 

injunction 

Agency must 
delay award or 
suspend 
performance 
unless CICA 
override 

Remedies 

Any 

appropriate 

actions 

Injunction or 
declaratory 
judgment if 
consistent with 
national 

; defense; bid or 
proposal costs 

Injunction or 
declaratory 
judgment, unless 
inconsistent 
with overriding 
public interest, 
j bid or proposal 
costs (limited 
to $10,000 under 
§1346(2)) 

Recommend that 
agency 
recompete, 
terminate, or 
award; contract 
costs; 
attorney's 
fees; bid or 
proposal costs 

i^?peals 

If within 
jurisdictional 
time period, 
begin action 
again in 
either GAO, 

COFC, or 
district court 

Federal Circuit 

Applicable court 

Indirectly 
through suit in 
the COFC or 
district court 


[Source: Ref. [6:p796] 
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1 . 


Protest Filed 


The first recourse available to a prospective 
contractor who feels unfairly treated is to file a formal 
written protest to the agency that issued the solicitation. 
This formal protest must contain, among other things, a 
detailed statement of the legal and factual grounds for the 
protest, including a description of resulting prejudice to 
the protestor. [Ref.2 :33.103(d)(2)] 

If the protest is based on alleged improprieties in a 
solicitation, and a protest is not filed prior to the 
closing date for receipt of proposals, such a protest will 
be considered "untimely filed" and will normally be denied. 
In all other cases, protests shall be filed no later than 
ten days after the basis of protest is known, or should 
have been known, whichever is earlier. Becoming aware of 
the basis for a protest normally occurs during the 
debriefing process. See Appendix B, pages 179-202 for a 
thorough understanding of the debriefing process. [Ref.2: 
33.103(e)] 

If the agency denies the protest, then the protestor 
has the option of pursuing the protest with the GAO. If 
the protestor decides take the protest to GAO, then he or 
she must file it not later than ten days after learning of 
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2. Agency Response 

Upon receiving a protest before award, unless the 
acquisition fulfills an urgent and compelling need or is 
otherwise in the Government's best interest, the 
contracting officer will withhold award pending resolution 
of the protest. 

If the protest is received within ten days after 
contract award, or within five days after a debriefing date 
offered to the protestor, the contracting officer shall 
suspend performance of the contract pending resolution of 
the protest. Here, too, the same exceptions of "urgent and 
compelling need" or "best interest of the Government" 
apply. [Ref: 2:33.103(f)]. 

The agency then must make every effort to resolve the 
agency-level protest within 35 days of receipt of the 
protest. 

3. Protestor Rebuttal 

A protestor who remains dissatisfied after resolution 
of his or her protest at the agency level may then file 
protest at GAO. From this date, the agency has 30 days to 
submit to GAO (with copies to the protestor and any other 
interested parties) a complete written report responding to 
the protest, including all relevant documents and an 
explanation of the agency's position. The protestor may 
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then file written comments on the agency report. Should 
the protestor fail to file comments, or, alternatively, 
request in writing that the case be decided on the existing 
written record, the case will be dismissed. [Ref.3:p.28-31] 

4. GAO Hearings and Decision 

GAO may, either at the request of a party or on its 
own initiative, conduct hearings in person or by telephone. 
Due to the increased costs and burdens they create, such 
hearings are held only when necessary. If GAO holds a 
hearing, all parties will be permitted to file written 
comments on the hearing. 

Once GAO has all the information necessary to decide 
the protest, it will issue a written decision through the 
Comptroller General within 100 days after the protest is 
filed. GAO'S decision will be to dismiss, deny or sustain 
the protest. The general criterion for GAO to sustain a 
protest is that the contracting agency violated procurement 
statutes or regulations. GAO will also recommend 
appropriate corrective action for those cases it sustains. 
Finally, GAO will recommend that the protestor be 
reimbursed for the costs of filing and pursuing the 
protest. [Ref. 3: p37-38] 

GAO's decisions are not binding, but are merely 
recommendations to which the contracting agency is not 
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required to adhere. The Comptroller General does, however, 
require agencies to respond in writing whenever they choose 
not to follow GAO's recommendations. The Comptroller 
General, in turn, reports to Congress annually on each 
instance where its recommendations were not fully 
implemented. In this way, the GAO exercises significant 
influence over the actions of the contracting agencies. 

[Ref. 3: p.38] 

C. CONCLUSION 

This chapter first provided a brief overview of the 
trade-off process. For a more comprehensive understanding 
of contracts awarded using the trade-off process, see 
Appendix B, which contains excerpts from the Army Material 
Command's pamphlet 715-3 entitled "Contracting For Best 
Value, A Best Practices Guide To Source Selection." This 
pamphlet is an excellent reference guide to the trade-off 
source selection process, from planning the procurement 
through debriefing offerors. 

Next, this chapter presented highlights of the protest 
process as it pertains to protests filed with the General 
Accounting Office. Competitive Negotiation, The Source 
Selection Process, by Cibinic, Nash and O'Brien, and Bid 
Protests At GAO: A Descriptive Guide, by the Office of 
General Counsel, GAO both are outstanding sources for a 
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comprehensive discussion of contract award controversies 
and the bid protest process at GAO, respectively. 
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III. QUANTITATIVE ANALYSIS OF PROTESTS FOR CALENDAR 
YEARS 1998 AND 1999 

A. INTRODUCTION 

This chapter provides an overview of the quantitative 
results of the data gathered from cases decided by the 
Comptroller General for calendar years 1998 and 1999. The 
chapter first presents the methodology used for collecting 
data. Then, it explains the rationale for narrowing the 
population to protests of contracts awarded using the 
trade-off process. Finally, the chapter provides 
statistics to give the reader an understanding of the 
breakdown of the sustained protests studied. 

B. DATA COLLECTION METHODOLOGY 

The global sample population for this study consisted 
of 558 protests decided by the Comptroller General between 
01 January 1998 and 31 December 1999. 

These cases were drawn from the LEXIS-NEXIS system by 
using the following queries within the Comptroller General 
sub-directory of the General Federal directory: 

• "Protest" AND "date>12/31/97" AND "date<7/l/98" 

• "Protest" AND "date>6/30/98" AND "date<l/l/99" 

• "Protest" AND "date>12/31/98" AND "date<7/l/99" 

• "Protest" AND "date>6/30/99" AND "date<l/l/00" 
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The researcher used four queries to extract the 
necessary data because the LEXIS-NEXIS system allows no 
more than 200 cases to be printed at one time. The 
resulting queries distributed the global population 
relatively evenly. 

From this global population, the researcher analyzed 
each case to determine if a trade-off process was used in 
evaluating solicitations and awarding the subject contract. 
This process narrowed the population to 325 cases. Appendix 
C provides a complete listing of these cases. 

Once these protests were identified, various data 
elements were captured for use in analyzing the cases 
further. In order to standardize this data collection, the 
researcher created a data collection form that contained 
the following data fields: 

• GAO case number 

• Case name 

• Decision date 

• Decision 

• Award Agency 

• RFP number 

• Reason for protest 

• Weaknesses identified 
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Next, the researcher selected protests that the 
Comptroller had either completely or partially sustained, 
resulting in a target population of 47 cases. Finally, the 
researcher eliminated five cases whose basis for protest 
did not meet his definition of an element of the trade-off 
process (e.g., competitive range determinations, small 
business set-aside decisions, etc.) to arrive at the final 
target' population of 42 cases. Figure 1 below provides a 
graphic representation of the narrowing of the global 
population down to the final target population. 
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FIGURE 1. DETERMINING STUDY TARGET POPULATION 
[Source: Developed by researcher.] 


C. METHODOLOGY FOR SELECTING THE PROTESTS OF CONTRACTS 

AWARDED USING THE TRADE-OFF PROCESS 

As stated above, the Comptroller General sustained 47 
protests of contracts awarded using the trade-off process 
during the time period studied. Of these, five were 
protests of peripheral issues rather than of the trade-off 
process itself and, therefore, were eliminated. Table 2 
lists the five cases that were eliminated from the target 
population, as well as the element protested. 
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TABLE 2. PROTESTS FAILING TO MEET DEFINITION 



[Source; 

: Developed by researcher.] 


Name 

Reason for protest 

1 

Aalco Forwarding 

RFP not properly set-aside for 

small business. 

2 

Hoechst Marion 

Russel, Inc. 

Requirements stated in RFP do not 

realistically state the Agency's 

requirements. 

3 

Technical Support 

Services, Inc. 

RFP did not state lowest priced, 

technically acceptable as award 

basis yet awarded as such. 

4 

Pemco Aeroplex, 

Inc. 

Unduly restrictive of competition 

due to combining requirements. 

5 

Dual, Inc. 

Awardee made misrepresentations in 

offer and agency relied upon these 

in its evaluation. 


D. OVERALL STATISTICS 

Of the 558 total protests for the calendar years 1998 
and 1999, 81 were sustained. The sustainment rate for 
award protests in general is, then, 14 percent (14.34 
percent). This compares to 325 protests of contracts 
awarded using the trade-off process, of which 47 were 
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ultimately sustained. The researcher did not include the 
remaining 233 protests filed because they were not protests 
of contracts awarded using the best value trade-off process 
as their source selection method. Rather, they were either 
awards that used sealed bidding or lowest 
priced/technically acceptable as their source selection 
method, or protests of issues not concerning the trade-off 
process in source selection (e.g., requests to reconsider a 
previous decision, requests to establish the amount to be 
reimbursed resulting from a previous decision, etc.). Of 
the 47 remaining in the target population, five did not 
involve the trade-off process. This gives a cumulative 
sustainment rate for trade-off process protests of 13 
percent (12.92 percent) for calendar years 1998 and 1999. 

Considering all protests of contracts awarded for 
calendar year 1998 independently, 328 were decided by the 
Comptroller General. Of these 328 protests, 48 were 
sustained, giving a sustainment rate of 15 percent (14.63 
percent) for all protests. 

By comparison, there were 196 protests of contracts 
awarded using the trade-off process. Of these, 168 were 
denied or dismissed, five were sustained, but did not 
involve the trade-off process, and 23 were sustained 
protests that challenged a trade-off process element. This 


28 



gives a sustainment rate of 12 percent (12.04 percent) for 
calendar year 1998. 

Similarly, there were a total of 230 protests in 
calendar year 1999. Of these, 33 were decided in favor of 
the protestor (sustained), providing a sustainment rate of 

14 percent (14.35 percent) for that year. 

This compares to a total of 129 protests of contracts 
awarded using the trade-off process. Of these, 19 were 
sustained and met the researcher's definition of protests 
of contracts awarded using the trade-off process. This, 
then, yields a sustainment rate for calendar year 1999 of 

15 percent (14.73 percent). Table 3 provides a summary of 
the sustainment rates for specific periods and protest 
classifications. Figure 2 graphically presents the 
information for both years studied. 
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TABLE 3. GENERAL AND SPECIFIC SUSTAINMENT RATES FOR PERIODS 

STUDIED 

[Source: Developed by researcher.] 


Category 

No. 

1998 

Of Cases and 

1999 

Percentages 

Combined 

Total cases 

328 

230 

558 

Total Sustained 

48/15% 

33/14% 

81/14% 

Trade-off 

process cases 

196 

129 

325 

Sustained trade¬ 
off process 

protests 

23/12% 

19/15% 

42/13% 
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(14%) Cases Protests 

(13%) 


Figure 2. PROTEST DATA FOR CY98/99 
[Source: Developed by researcher.] 


By further analyzing only the remaining 42 sustained 
protests, the researcher derived the following information. 
Analysis of the type of Government agency that awarded the 
contracts resulting in sustained protests provides the 
following data. Exactly half of the sustained protests were 
for contracts awarded by an Agency within the Department of 
Defense (DoD). The other half of the contracts were awarded 
by various other Federal agencies. However, considering 
the number of protests of contracts awarded using the 
trade-off process, the DoD had a lower sustainment rate 
than the other Federal agencies as a whole over both years— 
11 percent (11.11 percent) for the DoD compared to 15 











percent (15.44 percent) for all other Federal agencies. The 
remainder of the statistical analysis will treat these two 
groups—DoD and Other Federal Agencies (OFA) —as distinct 
segments of the target population for comparison purposes. 
Table 4 provides a tabular breakdown of this information. 


Table 4. Breakdown of protests and sustainment rates by DoD 
component compared to other Federal agencies. 

' Source. Developed by the researcher. 


DoD 

Other Fed. 
Total 


DoD 

Other Fed. 
Total 


DoD 

Other Fed. 
Total 


Number of 
Protests 
108 
90 
198 


Number of 
Protests 
81 
46 
127 


Number of 
Protests 
189 
136 
325 


1998 
Number 

Sustained 

9 

14 

23 

1999 
Number 

Sustained 

12 

7 

19 

Cumulative 

Number 

Sustained 

21 

21 

42 


Sustainment 

Rate 

8.33% 

15.56% 

11.62% 


Sustainment 

Rate 

14.81% 

15.22% 

14.96% 


Sustainment 

Rate 

11 . 11 % 

15.44% 

12.92% 
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E. STATISTICS BY AGENCY 

Analysis of the data within particular agencies 
provides the following results. Within the Department of 
Defense, the Navy accounted for nine sustained protests of 
trade-off awards for calendar years 1998 and 1999. As a 
percentage of all sustained protests of contracts awarded 
by the DoD using the trade-off process, the Navy awarded 43 
percent (42.86 percent) of the contracts for the period 
studied. Among protests of Navy contracts awarded using the 
trade-off process, there is a sustainment rate of 15 
percent (14.52 percent) for calendar years 1998 and 1999. 

Similarly, of the 21 sustained protests of contracts 
awarded using the trade-off process by DoD agenciesduring 
this period, the Army was responsible for six, or 29 
percent (28.57 percent). These numbers equate to a 
sustainment rate of ten percent (9.84 percent) for protests 
of Army contracts awarded using the trade-off process. 

Still, within the Department of Defense, two other 
agencies contributed to the DoD total with more than one 
sustained protest. The Air Force and the Defense Finance 
and Accounting Service (DFAS) each accounted for two 
sustained protests. The DFAS protests occurred in calendar 
year 1998 and accounted for 22 percent (22.22 percent) of 
the DoD total of nine sustained protests that year. With a 
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total of two protests of DFAS contracts awarded using the 
trade-off process in that, this produces a 100 percent 
sustainment rate. In calendar year 1999, the study reveals 
a single protest of a DFAS trade-off award that was 
ultimately denied by the Comptroller General. Similarly, 
the Air Force logged two sustained protests in calendar 
year 1999, which reflected 17 percent (16.67 percent) of 
the DoD total of 12. Given that there were 45 protests of 
Air Force trade-off contracts awarded for the two years 
studied, this gives a sustainment rate of four percent 
(4.44 percent). Table 5 provides a summary of these 
results. 
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Table 5. Suinmary of protests and protest sustainment 
rates within the Department of Defense. 
Source. Developed by the researcher. 


1998 


Air Force 
Army 
DFAS 
Navy 

other DoD 
DoD Total 


Air Force 
Army 
DFAS 
Navy 

Other DoD 
DoD Total 


Air Force 
Army 
DFAS 
Navy 

Other DoD 
DoD Total 


Number of 
Protests 
26 
37 
2 

32 

11 

108 


Number of 
Protests 
19 
24 
1 

30 

7 

81 


Number of 
Protests 
45 
61 
3 

62 

18 

189 


Number 

Sustained 

0 

2 

2 

4 

1 

9 

1999 

Number 

Sustained 

2 

4 
0 

5 
1 

12 


CUMULATIVE 

Number 

Sustained 

2 

6 

2 

9 

2 

21 


Sustainment 

Rate 

0 . 00 % 

5.41% 

100 . 00 % 

12.50% 

9.09% 

8.33% 


Sustainment 

Rate 

10.53% 

16.67% 

0 . 00 % 

16.67% 

14.29% 

14.81% 


Sustainment 

Rate 

4.44% 

9.84% 

66.67% 

14.52% 

11 . 11 % 

11 . 11 % 


Among Other Federal Agencies (OFA), only four of the 
13 agencies had more than one sustained protest- These 
were the Department of Health and Human Services (HHS), the 
Department for Housing and Urban Development (HUD), the 
Department of Agriculture (DoA) and the General Services 
Administration (GSA). 

The Departments of Health and Human Services and 
Housing and Urban Development topped the list with four 
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sustained protests each. This equates to 19 percent (19.05 
percent) of all sustained protests within the OFA sub¬ 
group. The difference between the two agencies, in terms 
of sustained protests, lies in the number of protests filed 
against them—ten for HHS and 11 for HUD. This yields 
sustainment rates of 40 percent (40.00 percent) and 36 
percent (36.36 percent), respectively. The next Federal 
agency that experienced more than a single sustained 
protest was the Department of Agriculture (DoA), which had 
three. That accounted for 14 percent (14.29 percent) of 
all OFA sustained protests and gave DoA a sustainment rate 
of 20 percent (20.00 percent), given that there were 15 
protests filed against their awarded contracts for the 
relevant period. The only other Federal agency to receive 
more than a single unfavorable decision from the 
Comptroller General in response to a filed protest was the 
General Services Administration (GSA). GSA had two of the 
15 protests brought against it sustained, or 13 percent 
(13.33 percent). As a percentage of all OFA sustained 
protests, GSA accounted for ten percent (9.52 percent). 
Table 6 depicts the number of protests, as well as the 
number of those sustained, among agencies other than the 
DoD, for the subject calendar years. 
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Table 6. Summary of protests and protest sustainment rates 
among other Federal agencies. 

Source: Developed by the Researcher 


DoA 

GSA 

HHS 

HUD 

Misc. OFA 
OFA Total 


DoA 

GSA 

HHS 

HUD 

Misc. OFA 
OFA Total 


DoA 

GSA 

HHS 

HUD 

Misc. OFA 
OFA Total 


Number of 
Protests 
10 
9 
9 
7 

56 

90 


Number of 
Protests 

5 

6 
1 
4 

30 

46 


Number of 
Protests 
15 
15 
10 
11 
86 
136 


1998 
Number 

Sustained 

3 
0 

4 
2 
6 

14 

1999 
Number 

Sustained 

0 

2 

0 

2 

3 

7 


CUMULATIVE 

Number 

Sustained 

3 
2 

4 
4 
9 

21 


Sustainment 

Rate 

30.00% 

0 . 00 % 

44.44% 

28.57% 

10.71% 

15.56% 


Sustainment 

Rate 

0 . 00 % 

33.33% 

0 . 00 % 

50.00% 

10 . 00 % 

15.22% 


Sustainment 

Rate 

20 . 00 % 

13.33% 

40.00% 

36.36% 

10.47% 

15.44% 


F. CONCLUSION 

This chapter displays the quantitative results of the 
researcher's analysis. The overall sustainment rate of 
protests of contracts brought before the Comptroller 
General was 14 percent for calendar years 1998 and 1999 
cumulatively. By comparison, the sustainment rate for 
protests of contracts awarded using a trade-off process for 
the same period was 13 percent. In other words, whether a 
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contract was awarded using the LPTA approach, sealed bids, 
or the trade-off process, there was virtually no difference 
in the rate at which protestors received a favorable 
decision. 


38 



IV. QUALITATIVE ANALYSIS OF PROTESTS FOR CALENDAR YEARS 

1998 AND 1999 

A. INTRODUCTION 

This chapter will provide a qualitative analysis of 
the 42 cases of protests of contracts awarded using a 
trade-off process, which the Comptroller General decided 
during calendar years 1998 and 1999. First, the chapter 
presents summary statistics for the eight categories of 
sustaining elements observed among the cases examined. 

Then, it presents a thorough discussion of each category. 
Additionally, whereas Palmer [Ref.7] examined all the 
sustained cases for the single year studied, this 
researcher will limit discussion to those cases that best 
portray each of the sustaining element categories. As a 
side note, the researcher observed three instances in which 
one of the sustaining elements cited by the Comptroller 
General was not an element relevant to the trade-off 
process (e.g., improper agency statement of need, improper 
agency treatment of competing offerors, etc.). These 
instances will not be discussed in this chapter; nor were 
they included in the computation of the statistics 
presented in the next section. 


39 



B. SUMMARY STATISTICS 

For the period studied, the Comptroller General's most 
frequently cited reason for sustaining protests was that 
either the Source Selection Evaluation Board (SSEB) or the 
Source Selection Authority (SSA)—or both—had not 
adequately documented their case files in support of the 
award recommendation or the award decision, respectively. 
This was a sustaining element in 17 of the 42 cases 
analyzed, or 40 percent (40.48 percent). 

Considering the total number of sustaining elements—71 
in all—cited by the Comptroller General for all 42 cases, 
this particular element was noted 24 percent (23.94 
percent) of the time. 

Occurring slightly less often was the judgment that 
the evaluation of proposals in response to a particular 
solicitation had not been conducted in accordance with the 
stated evaluation scheme of the Request For Proposals 
(RFP). This was a sustaining element in 14 cases, or 33 
percent (33.33 percent). As a whole, this represented 20 
percent (19.72 percent) of all sustaining elements passed 
down in Comptroller General decisions for the period 
studied. 

The next most frequent cause of sustained protests was 
an improper evaluation by the awarding agency of a 
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proposal's cost or price. This occurred in 12 of the 42 
cases studied, or 29 percent (28.57 percent). The 
frequency of this cause relative to all causes, according 
to the researcher's analysis, was 17 percent (16.90 
percent). 

Next, two sustaining elements occurred with equal 
frequency. The first was that the awarding agency 
improperly evaluated an offeror's past performance when 
conducting trade-offs; the other was that an agency failed 
to conduct meaningful discussions with offerors, as 
required by the Federal Acquisition Regulation (FAR). 

These sustaining elements each occurred in- nine cases, or 
21 percent (21.43 percent) of the cases analyzed. 

Another frequently occurring sustaining element was an 
awarding agency's improper evaluation of the technical 
merit of an offeror's proposal. This was a sustaining 
element in seven, or 17 percent (16.67 percent), of the 42 
sustained protests of contracts awarded using the trade-off 
process. As a function of all elements cited by the 
Comptroller General in sustaining the protests studied, 
this element accounted for ten percent (9.86 percent). 

In two cases, the Comptroller General noted that an 
awarding agency had improperly evaluated the labor 
qualifications of key personnel proposed by offerors in 
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particular cases. This equates to five percent (4.76 
percent) of the 42 cases studied and three percent (2.82 
percent) of all sustaining elements cited by the 
Comptroller General. 

Finally, there was a single case in which the awarding 
agency had failed to adequately disclose the factor 
evaluation weights in the solicitation. This one case 
accounted for two percent (2.38 percent) of all relevant 
cases for the period studied. Additionally, the single 
occurrence of this particular sustaining element equates to 
a mere one percent (1.41 percent) of all elements cited by 
the GAO in the subject cases. 

The eight sustaining elements, and the frequency with 
which each occurs, are summarized in Table 6. Figure 5 
presents the same information graphically. 
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Table 7. Sustaining elements and frequency 
Source. Developed by the researcher 


Sustaining Element 

Frequency 

Percent of 

Total Cases 

Selection Decision not 

Adequately Documented 

17 

40% 

^Evaluation of Proposals 

Inconsistent with RFP 

14 

33% 

Improper Agency Evaluation 

of Cost/Price 

12 

29% 

Improper Agency Evaluation 

of Past Performance 

9 

21% 

Agency Improperly Failed to 

Conduct Meaningful 

Discussions 

9 

21% 

Improper Agency Evaluation 

of Technical Merit 

7 

17% 

Improper Agency Evaluation 

of Labor Qualifications 

2 

5% 

Failure to Adequately 

Disclose Factor Evaluation 

Weights in Solicitation 

1 

2% 
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Legend: 


A - Selection decision not adequately documented 
B - Evaluation of proposals inconsistent with RFP 
C - Improper agency evaluation of cost/price 
D “ Improper agency evaluation of cost/price 

E - Agency improperly failed to conduct meaningful discussions 
F - Improper agency evaluation of technical merit 
G - Improper agency evaluation of labor qualifications 

H - Failure to adequately disclose factor evaluation weights in solicitation 

r\ 

^ r ‘ y 

■ Figure 3. Sustaining elements as a percent of total 
/.IV cases analyzed 

Source. Developed by the researcher 











It is interesting to note that, in 23 cases, (54.76 


percent) the protest was sustained based on a single 
sustaining element and that, in six cases, GAO sustained 
the protest citing three or more sustaining elements. In 
fact, the Comptroller General cited four or more sustaining 
elements in three decisions (7.14 percent). This is not 
terribly significant since it takes only one proven flaw 
for GAO to sustain the protest, but it does indicate that a 
small percentage of cases are plagued by many problems. 
Figure 6 graphically presents the distribution of the 
various combinations of sustaining elements. 



Figure 4. Sustaining Element Combination 
Source. Developed by the researcher 


C. SUSTAINING ELEMENT CATEGORIES 

1. Selection Decision Not Adequately Docixnaented 

The following subsections will present two sustained 
protests of contracts awarded using a trade-off process in 
which the selection decision was not adequately documented. 
This failure to document the trade-offs made in the 
decision process was a fatal flaw, which ultimately led to 
the offeror's protest being sustained. Additionally, the 
researcher will make recommendations for Federal 
contracting officers to consider in order to mitigate the 
risk of suffering sustained protests due to inadequately 
documented selection decisions. 

a. Bzospberics Incorporated 

Biospherics Incorporated; B-278508.4, 8-278508.5, 
B-278508.6; Comptroller General of the United States; 6 
October 1998, (Biospherics) is a protest in which agency 
failure to adequately document its selection decision was 
flawed. This flaw led GAO to conclude that it was not 
possible to determine the reasonableness of the agency 
evaluation upon which the award decision was made. 
Biospherics protested the Agency for Health Care Policy and 
Research (AHCPR), Department of Health and Human Services 
award of a contract to Logistics Applications, Inc, (LAI) 
for the operation of a publication clearing house. The 
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Request For Proposals (RFP) contemplated the award of a 
cost-plus-fixed-fee contract for the base period and four 
one-year option periods. The RFP required the contractor 
to store and distribute AHCPR publications; to maintain and 
manage AHCPR's automated mailing/inventory control systems; 
and to manage the storage and shipping of AHCPR exhibits. 
Award was to be made to the best advantage of the 
Government, cost and other factors considered. 

The following technical evaluation factors were 
contained in the RFP: 1) understanding the problem; 2) 
technical approach; 3) management plan; 4) key personnel; 
and 5) facilities. Each offer was to be evaluated, and a 
recommendation concerning technical acceptability was to be 
made by a peer review technical committee. Offers 
determined to be technically acceptable would then be 
evaluated for past performance. The RFP stated that 
technical merit was a paramount consideration and that cost 
would become a significant factor only if two or more 
proposals were determined to be technically equal. 

A peer review panel consisting of three 
individuals evaluated proposals submitted by three firms. 
All three firms were included in the competitive range, and 
the panel held discussions that focused on technical and 
cost issues. The offerors submitted final proposal 
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revisions. However, the agency made no adjustments to any 
of their technical scores; nor did it produce any 
documentation reflecting an analysis of the revised 
proposals. AHCPR selected LAI as the most advantageous 
offeror since it submitted the highest-technically-rated, 
lowest-proposed-cost proposal. 

Biospherics protested, raising two challenges to 
AHCPR's evaluation of its proposal: first, that its 
proposal should have been rated technically superior to 
LAI's; and second, that the agency decided to award a 
contract to LAI as a technically superior offeror. For the 
purposes of this discussion, the two challenges will be 
treated as one—that Biospheric's proposal was technically 
superior and should have received the award. 

The agency countered by stating that Biospheric's 
final proposal revision represented an improvement in its 
technical proposal, but not enough to be considered 
technically equal with LAI. When GAO asked the agency to 
point out where the record documented a comprehensive look 
at the final results of the technical, cost and past 
performance evaluations, the agency responded by asserting 
that the source selection memorandum represented the 
comprehensive assessment of those factors. The agency also 
stated that the source selection memorandum directly 
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reflected a comprehensive discussion of the relative 
technical strengths and weaknesses of the three proposals, 
although that discussion was not recorded in writing. 

GAO repeatedly states in its write-up of protest 
decisions that it will not reevaluate proposals, but, 
rather, will examine the record to determine whether the 
agency's judgment was reasonable and in accordance with the 
RFP's stated evaluation criteria. 

Further, it is a basic principle of GAO that, 
while adjectival ratings and point scores are useful as 
guides to decision-making, they generally are not 
controlling. Instead, they must be supported by reasoned 
documentation of the relative differences between proposals 
and the basis for the selection decision. 

The lack of reasoned documentation was the fatal flaw in 
this case. There was no discussion regarding the relative 
strengths and weaknesses of the proposals and no post¬ 
discussion narrative supporting the agency's conclusion. 
Because of this lack of documentation, GAO found it could 
not determine the reasonableness of the agency's selection 
decision. The Comptroller General sustained the protest on 
these grounds and recommended that AHCPR reevaluate the 
proposals, document its evaluation and make a new selection 
decision. [Ref. 8] 
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Jb. Si-Nor Incorporated 

Si-Nor Incorporated; B-282064, B-282064.2; 
Comptroller General of the United States; 25 May 1999, (Si- 
Nor) is a protest in which the agency record does not 
establish that the trade-off decision was reasonable. Si- 
Nor protested the Air Force award of a fixed-price 
requirements contract for an eight-month base period, with 
four one-year options. The contract was for refuse 
collection and disposal services at Hickam Air Force Base, 
Hawaii, with award to be made on a best-value basis. 

The RFP required no technical proposals; rather, 
each offeror was required to submit financial references 
and at least three past performance references for service 
contracts of a similar size in which it was the prime 
contractor. Proposals were to be evaluated based on: 1) 
past performance/experience; and 2) price/cost, which were 
approximately equal in weight. Past performance and 
experience were also to be approximately equal in weight, 
with past performance to be assessed to determine the 
offeror's relative reliability to perform the contract 
requirements. Experience was to be evaluated to assess the 
offeror's experience in performing work on similar types of 
service contracts. 
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Of the five offers received, the Air Force 
determined that the proposal from U.S. Eagle, Incorporated 
(Eagle) represented the best value to the Government based 
on its record of outstanding past performance. The Air 
Force believed that Eagle was worth the nearly $900,000 
price premium over Si-Nor's proposal. 

Si-Nor protested on the basis that the agency 
failed to document why any advantage reflected in Eagle's 
past performance rating warranted the high price premium. 

In its judgment, GAO cited its ruling in the 
matter of Dyncorp; B-245289, B-245289.2; Comptroller 
General of the United States; December 23, 1991, in which 
it determined that: Price/technical trade-offs may be made, 
and the extent to which one may be sacrificed for the other 
is governed only by the tests of rationality and 
consistency with the established evaluation criteria. 

Further, in the matter of TRW, Inc.; B-234558; 
Comptroller General of the United States; 21 June 1989; GAO 
stated: 

In deciding between competing proposals, the 
propriety of such a trade-off turns not only on 
the difference in technical scores or ratings per 
se, but on whether the selection officials 
judgment concerning the significance of that 
difference was reasonable and adequately 
justified in light of the RFP evaluation scheme. 


51 


Based on these prior rulings and the record of 
this particular case, GAO decided in favor of the 
protestor. The Comptroller General ruled that, while the 
record clearly states the agency's conclusion that Eagle's 
past performance rating was worth the price premium, it 
includes no documentation indicating why its past 
performance rating would outweigh the additional price. 

The Comptroller General recommended that the Air Force 
perform and document a proper trade-off analysis. [Ref. 9] 
c. Element Summary 

The two cases discussed above are typical of the 
remainder of the protests sustained during the period 
studied that contained this particular sustaining element. 

Contracting officers must not only perform a 
trade-off analysis that incorporates sound business 
judgments, but they must also adequately support their 
decisions with reasoned documentation. This documentation 
must include the benefits associated with awarding a 
contract to offerors whose proposals represent a price 
premium with respect to other technically acceptable 
offers. 

2. Evaluation of Proposals Inconsistent with RFP 

The second most frequently observed sustaining element 
was the evaluation of proposals in a manner that deviated 
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from the stated evaluation scheme of the RFP. This flaw in 


the evaluation process creates a bias in the selection 
decision that, although it may be unintentional, is most 
certainly detrimental to the Government's reputation as an 
even-handed partner in its business dealings. The two 
cases discussed below contained this category of sustaining 
element, although, as in previously discussed cases, it was 
not the only sustaining element in the protest. 

a. Marquette Medical Systems, Inc. 

Marquette Medical Systems, Inc.; B-277827.5, B- 
277827.7; Comptroller General of the United States; 29 
April 1999, (Marquette) is a protest in which the Federal 
agency improperly deviated from its solicitation evaluation 
criteria in assessing competing proposals. Marquette 
protested the Army's award of a contract to Hewlett-Packard 
Company for a Cardiology Medical Information System (CMIS) 
to replace the Department of Defense's (DoD) existing 
computerized medical information system. 

The RFP anticipated the award of a fixed-price 
contract for a base period of one year, with options for 
seven additional years. The solicitation requested 
proposals for providing hardware, software, documentation, 
training, support and maintenance services relating to 
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storage, and retrieval and interpretation of 
electrocardiograms (ECG). The RFP stated that there were 
950 ECG carts in the existing system and invited offers 
based upon either replacing or upgrading the cart 
inventory. 

Award was to be made to the responsible offeror 
whose proposal, conforming to the solicitation, was 
determined to be the best overall value, price and other 
factors considered. The technical evaluation criteria, in 
descending order of importance (factors three and four were 
comparatively equal) were: 1) quality of technical 
approach; 2) contract management; 3) understanding of 
overall contract requirements; (4) past performance and 
relevant experience; and (5) key personnel qualifications. 
The Army received three proposals and ultimately determined 
that Hewlett-Packard offered the best value. 

Marquette protested the award decision, alleging 
that the agency's evaluation of proposals was unreasonable 
and inconsistent with the RFP's evaluation scheme. A 
review of the record shows that the contracting officer 
considered the cost of purchasing new ECG carts over the 
potential eight-year life of the contract. The contracting 
officer reported that, since Hewlett-Packard proposed 
replacing the ECG carts while Marquette offered to upgrade 
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them, he "normalized" the two offers by deducting a certain 
amount from Hewlett-Packard's offer and adding on to the 
cost of Marquette's offer. The contracting officer intended 
this "normalization" to account for the minimal need to 
acquire new carts during the contract life for the Hewlett- 
Packard offer, as well as the additional cost of replacing 
carts over the contract period if Marquette's proposal were 
accepted. 

The flaw in the contracting officer's logic, as 
GAO noted, was that, while the RFP indicated that a price 
analysis would be conducted, it gave no indication that the 
analysis would involve deducting the price for performing 
any part of the work (and especially not the price for a 
major component such as the ECG carts) from an offeror's 
total price in an attempt to "normalize" the competition. 

In fact, as the Comptroller General correctly noted in the 
matter of Sisters of Charity of the Incarnate World; B- 
251799 et.al.; Comptroller General of the United States; 4 
May 1993: "where the contract is fixed price, an agency is 
not permitted to make adjustments for cost elements, since 
the fixed price is the price the awardee is obligated to 
perform at and the price the Government is bound to pay." 

The Comptroller General sustained this protest 
based on the fact that once offerors are informed of the 
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criteria against which their proposals will be evaluated, 
the agency must adhere to those criteria or inform all 
offerors of any significant changes made in the evaluation 
scheme. Accordingly, GAO recommended that the Army 
reconsider whether the RFP's current evaluation scheme 
represented its needs and, if so, to reevaluate proposals. 
[Ref.10] 


Jb. Opti-Lite Optical 

Matter of Opti-Lite Optical; B-281693; 

Comptroller General of the United States; 22 March 1999, 
(Opti-Lite) is a protest in which the award determination 
was improperly based on a comparison of total technical and 
price point scores under a solicitation that did not 
provide for such a point-based trade-off. Opti-Lite 
protested the Department of Veterans Affairs (VA) award of 
an indefinite-delivery, indefinite-quantity requirements 
contract to Classical Optical Laboratories (Classical) for 
commercial item prescription eyeglasses and services for VA 
beneficiaries. 

The solicitation provided for award to the 
responsible offeror whose conforming offer was most 
advantageous to the Government. The stated technical 
evaluation criteria consisted of: 1) methodology of 
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approach; 2) personnel qualifications; and 3) past 
performance. The solicitation further provided that 
technical and past performance combined were approximately 
equal in weight to price. Of the six proposals submitted. 
Classical's was selected as being most advantageous to the 
Government. 

Opti-Lite protested the VA's decision, alleging, 
among other things, that the agency's decision not to allow 
alternate pricing proposals was a major change requiring 
the agency to again request final proposal revisions. The 
Comptroller General's discussion of this protest revolves 
around two issues: a mechanical application of the 
price/technical trade-off methodology; and the fact that 
the contracting officer apparently made no qualitative 
assessment of the proposals as required by the evaluation 
scheme stated in the RFP. In sustaining the protest, the 
Comptroller General stated that the award determination was 
based entirely on a comparison of total technical and price 
point scores under a solicitation that did not provide for 
such a point-based trade-off. [Ref. 11] 

c. Element Smamary 

These sustained protests are typical of the 
remaining twelve that cited as a sustaining element an 
agency's deviation from the evaluation scheme stated in the 
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RFP. Where an agency has been unwilling or unable to 
adhere to the prescribed method, the Comptroller General 
has sided with the protestor in every instance. The lesson 
for contracting officers is that they must abide by the 
stated evaluation criteria of an RFP, or, at the very 
least, advise all offerors of a required deviation in order 
to allow them to revise their proposals. The clear and 

f 

convincing penalty for doing otherwise is to invite the 
risk of a sustained protest. 

3. Improper Agency Evaluation of Cost/Price 
This subsection presents two cases in which an 
agency's improper evaluation of cost/price resulted in 
protest sustainment. Contracting officers must determine 
not only that a proposal's cost is fair and reasonable, but 
also what a contract should cost, assuming reasonable 
economy and efficiency. 

a. Tidewater Construction Corporation 
Tidewater Construction Corporation; B-278360; 
Comptroller General of the United States; 20 January 1998, 
(Tidewater) is a protest in which the agency failed to 
perform a proper cost realism analysis. Its only 
evaluation of the offerors' proposals was a comparison 
between the proposed costs for various work items and the 
government estimate. Tidewater protested the U.S. Navy 
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award of a cost-plus-award-fee contract for the 
construction of a crane rail interconnect system to 
Pizzagalli Construction Company. 

The RFP evaluation scheme provided for award on a 
best-value basis in which the technical factors combined 
were to be more important than cost. Proposals were to be 
based on a 60-percent design of the project. The RFP did 
not request detailed cost data, but did advise offerors 
that supporting documentation for large elements of cost 
would be requested during discussions. 

The Navy hired an architect/engineer to prepare 
cost estimates tracking 35-percent, 60-percent, and 100- 
percent design of the project in order to assist in the 
cost evaluation. All proposals received were included in 
the competitive range. Following discussions, offerors 
submitted final proposal revisions incorporating, among 
other things, the 100-percent design of the project. The 
Navy's cost realism evaluation consisted solely of 
comparing each offeror's target cost to the government 
estimate. 

Tidewater protested the Navy's evaluation of cost 
realism and alleged that the government estimate was 
defective. Defense Federal Acquisition Regulation 
Supplement sections 215.801 and 215.805-70 require that. 
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whenever a cost reimbursement contract is anticipated, a 
cost realism analysis be performed to determine if the 
overall costs in an offeror's proposal: 1) are realistic 
for the work to be performed; 2) reflect a clear 
understanding of the requirements; and 3) are consistent 
with the various elements of the offeror's technical 
proposal. In considering this protest, GAO noted that its 
review of an agency's cost evaluation is limited to 
determining whether the evaluation was reasonably based and 
was not arbitrary. 

In sustaining this protest, the Comptroller 
General found that the Navy, by determining cost realism 
based solely on a comparison between the offeror's target 
costs and the government estimate, acted improperly. GAO 
further stated that the agency's evaluation was a price 
analysis intended only to ensure a fair and reasonable 
price and could not reasonably be used to ascertain the 
cost realism of a cost reimbursement proposal. The reason 
for this is that the Government will be liable for the 
contractor's allowable and allocable costs, not the 
proposed costs. A cost realism analysis allows the 
Government to determine what the contract should cost, 
assuming reasonable economy and efficiency. GAO 
recommended that the Navy obtain cost data sufficient to 
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perform a proper cost realism analysis and make a new 
source selection. [Ref. 12] 

Jb. E.L. Hamm & Associates^ Inc. 

E.L. Hamm & Associates,’ Inc.; B-280766.3; 
Comptroller General of the United States; 12 April 1999, 
(Hamm) is a protest where the agency did not perform a 
proper cost realism evaluation. Hamm protested the U.S. 
Army award of a cost-plus-award-fee contract, for a base 
period with four option years, for storage and warehouse 
services at the Army Training Support Center (ATSC), Fort 
Eustis, Virginia. 

The RFP requested that offerors submit separate 
technical/management and cost proposals corresponding to 
four evaluation factors: 1) technical; 2) management; 3) 
past performance; and 4) cost. The Army determined that the 
proposal submitted by Communication Technologies, Inc. 
(Comtek) represented the best overall value to the 
Government. 

Of the proposals received, six were included in 
the competitive range. Discussions were conducted and 
final proposal revisions requested from all six offerors. 
The evaluators concluded that the added benefits offered by 
the technically superior, highest-ranked proposal were not 
worth the cost premium and, therefore, recommended award to 
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Comtek as the offeror with the next highest-ranked, 
technically superior offer with the lowest evaluated cost. 
Hamm filed a protest alleging that the agency's cost 
realism evaluation was flawed, partly because Comtek's 
proposed overhead rates were unrealistically low. 

The record does not justify the agency's 
determination that Comtek's proposed overhead rate was 
realistic. The contracting officer's rationale for 
accepting this rate, which is significantly lower than the 
one approved by the Defense Contract Audit Agency, is 
unfounded. Indeed, the contracting officer stated that the 
agency accepted Comtek's proposed rate without any cost 
realism evaluation. For these reasons, the Comptroller 
General found that the agency's actions prejudiced Hamm 
and, therefore, sustained the protest. Among its 
recommendations, the GAO stated that the agency must 
perform a proper cost evaluation, including a cost realism 
analysis. [Ref. 13] 

c. Element Summary 

These findings by the Comptroller General point 
to the importance of a proper cost/price evaluation. The 
Federal Acquisition Regulation section 15.305 states that, 
when contracting on a fixed-price type basis, adequate 
competition will usually establish price reasonableness. 
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For cost reimbursement contracts, evaluation must include a 
cost realism analysis to determine not only the cost 
realism of the offeror's proposal, but also the cost the 
Government should realistically expect to pay, assuming 
reasonable economy and efficiency. [Ref. 2:15.305(1)] 

Not only must contracting officers determine 
cost/price reasonableness—through either competition or a 
cost realism analysis—but they must also document this 
determination. This goes back to the researcher's most 
commonly observed sustaining element—that is, performing a 
cost realism analysis without documenting the results is 
effectively the same as not performing the analysis in the 
first place. 

4. Improper Agency Evaluation of Past Perfonaance 

The two cases discussed below involve protests that 
were sustained solely because the contracting agency 
improperly evaluated the offeror's past performance. Aside 
from the two cases presented, this sustaining element was 
cited by the Comptroller General in seven other cases among 
the 42 studied. 

a. U.S. Property Management Service Corporation 

U.S. Property Management Service Corporation; B- 
278727; Comptroller General of the United States; 6 March 
1998, (U.S. Property) is a protest in which improper agency 
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evaluation of past performance was the basis for a 
sustained argument. U.S. Property protested the Department 
of Housing and Urban Development (HUD) award of a small 
business set-aside contract for real estate asset 
management services for single family properties owned by 
HUD in its District of Columbia office jurisdiction. HUD 
determined that a proposal by Ernie Stefkovic Associates, 
Inc. (ESA) represented the best value to the Government. 

The RFP provided for a cost/technical trade-off 
and stated that the combined weight of the technical 
evaluation factors was more significant than price. The 
RFP listed the technical evaluation factors, with their 
relative weights, to be scored on a 100-point scale, as 
follows: 

• Prior management experience 30 points 

• Past performance 25 points 

• Management capability 25 points 

• Office location(s) 20 points 

Offerors were required to supply references for 
recent, relevant contract work under the prior management 
experience and past performance factors. The incumbent 
contractor did not submit a proposal since it was not a 
small business. The presidents of U.S. Property and ESA 
were former managers of the incumbent who formed their own 
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property management company in 1994. Due to a dispute 
between the principals, these two individuals broke away 
from this company. The president formed ESA, while the vice 
president formed U.S. Property, for the purpose of 
submitting independent proposals in response to HDD's RFP. 

Since the corporations were newly formed, the 
presidents of both ESA and U.S. Property drew upon their 
shared experience with the incumbent contractor to 
establish their qualifications under the prior management 
experience and past performance factors. 

HUD included 12 proposals in its competitive 
range, including ESA's and U.S. Property's. After written 
discussions, it further narrowed the field to five 
proposals, still including ESA's and U.S. Property's. The 
record shows that ESA's proposal was rated higher than U.S. 
Property's, primarily due to the prior management 
experience factor. The Technical Evaluation Panel (TEP) 
downgraded U.S. Property's proposal because it was a newly 
formed corporation that lacked independent corporate 
experience. The TEP did not similarly question ESA's lack 
of independent corporate experience. 

In its revised proposal, U.S. Property provided 
evidence that it had recently acquired independent 
corporate experience, having been awarded two contracts for 
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property management services. The TEP subsequently 
increased the protestor's prior management experience score 
slightly. Ultimately, the TEP awarded significantly higher 
scores for prior management experience and past performance 
to ESA than it did to U.S. Property. 

Based on the TEP's recommendation and the source 
selection authority's belief that ESA possessed more 
corporate experience than U.S. property, HUD awarded the 
contract to ESA. U.S. Property protested on the grounds 
that the agency had conducted an unequal evaluation with 
respect to prior corporate experience. 

The Comptroller General's review of the record 
confirmed that HUD did not evaluate proposals equally under 
either the prior corporate experience or the past 
performance factor. The TEP consistently downgraded the 
protestor's proposal in these two areas, yet could 
demonstrate nothing in the record to explain why the same 
judgment did not apply to ESA, especially since ESA's 
proposal reflected no corporate experience beyond its 
principal's shared experience with the president of U.S. 
Property. 

Finally, the Comptroller General judged that 
ESA's high scores stemmed from the agency's evaluation of 
its incxambent personnel, not from its corporate experience. 
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which was the standard by which it had evaluated all other 
proposals. 

Based on these findings, the Comptroller General 
sustained the protest and recommended that the agency 
reevaluate the final proposal revisions. [Ref. 14] 

Jb. Nova Groupf Inc. 

Nova Group, Inc.; B-282947; Comptroller General 
of the United States; 15 September 1999, (Nova) is a 
protest in which the agency improperly downgraded an 
offeror's past performance based merely on that offeror's 
history of contract claims. Nova protested the U.S. Navy's 
award of a fixed-price construction contract for two 
projects, with options for eight additional projects, at 
the Naval Station, Pearl Harbor, Hawaii. The Navy awarded 
to Healy Tibbitts Building, Inc. as the offeror submitting 
the best value proposal. 

The solicitation stated that its trade-off 
evaluation plan gave equal weight to technical evaluation 
and price. The RFP listed the technical evaluation 
factors, in descending order of importance, as follows: 

• Construction management plan 

• Experience/past performance 

• Experience of key personnel 

• Small business subcontracting plan 
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With regard to the experience/past performance 
factor, Nova's proposal was rated satisfactory, rather than 
outstanding, because its proposal had not clearly addressed 
the disposition of a number of contract claims. The 
Technical Evaluation Board (TEB) determined that Nova's 
claims record did not demonstrate a general trend for 
outstanding past performance in terms of customer 
satisfaction. Nova protested the evaluation of its 
proposal under the first two evaluation factors. 

Discussion of this protest will be limited to the 
evaluation of Nova's experience/past performance since the 
Comptroller General denied the first protest issue. The 
protestor alleged that the Navy improperly considered 
Nova's contract claims as the basis for downgrading its 
rating. 

GAO notes that, absent some evidence of abuse of 
the contract dispute process, contracting agencies should 
not downgrade an offeror's proposal simply because it has 
filed claims. GAO continues by noting the matter of Amclyde 
Engineered Products Co.r Inc.; B-282271, B-282271.2; 
Comptroller General of the United States; 21 June 1999, in 
which it found that contract claims constitute remedies 
established by statute and that firms should not be 
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prejudiced because of their reasonable pursuit of such 
remedies in the past. 

Since Nova's actual past performance had been 
rated "average to outstanding" by previous contracting 
agencies, the Navy's only basis for downgrading Nova's 
proposal was the existence of nine previous contract 
claims. The Comptroller General ruled that the Navy's 
evaluation penalized Nova simply for using the contract 
dispute process and, therefore, sustained the protest. 
Consequently, the Comptroller General recommended that the 
Navy reopen discussions, evaluate proposals consistent with 
its decision, and make a new source selection decision. 

[Ref. 15] 

c. Element Svamaary 

As the above two cases illustrate, contracting 
officers have an obligation to equitably and fairly 
evaluate past performance as an indicator of an offeror's 
ability to perform a contract successfully, if the 
solicitation calls for such an evaluation. The 
solicitation must also explain how offeror's proposals 
without relevant past performance will be evaluated. The 
FAR specifically states that offerors without a record of 
past performance will not be evaluated favorably or 
unfavorably on past performance. [Ref. 2:15.305(2)] 
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5. Agency In^roperly Failed 1:0 Conduct Meaningful 
Discussions 

Biospherics, Inc. and Mechanical Contractors, S.A. are 
just two of nine cases that were sustained due to an 
agency's failure to properly conduct meaningful 
discussions. As the researcher will show, if an agency 
conducts discussions with offerors, such discussions must 
be meaningful and must inform offerors of the areas their 
proposals that require amplification or correction. To do 
otherwise increases the risk of a sustainable protest, 
a. Biospherics, Inc. 

Biospherics, Inc.; B-278278; Comptroller General 
of the United States; 14 January 1998, (Biospherics) is a 
protest in which the agency conducted prejudicially 
inadequate and misleading discussions. Biospherics 
protested the U.S. Forest Service's award of a fixed-price, 
requirements contract for the development, implementation, 
and operation of the National Recreation Reservation 
Service. The RFP advised offerors that award would be 
based on a cost/technical trade-off and that technical 
evaluation factors were more important than price. The 
contract was to be for a 9.5 year period. The Forest 
Service found that Park.net, Inc.'s (Park.Net) proposal 
best represented overall value to the Government. 
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The Forest Service held written and oral 


discussions with three offerors in the competitive range. 
Since all three proposals contained unit pricing that was 
higher than the independent Government estimate, the agency 
advised them to review their cost information to determine 
the best pricing scheme they could offer. After more 
discussions and upon reviewing the final proposals of all 
offerors in the competitive range, the agency made award to 
Park.Net, citing Biospherics' proposed prices as 
unrealistically low. Biospherics alleged that the agency's 
discussions regarding Biospherics' proposal prices were 
inadequate and misleading. Specifically, Biospherics 
alleged that the agency not only failed to inform them that 
it viewed Biospherics pricing as unrealistically low, but 
also, following the first round of discussions, stated that 
the firm should review its proposal "for any additional 
savings." The agency countered by stating that it found 
Biospherics' prices unrealistically low only after 
reviewing the final proposal revisions. The agency 
admitted, however, that it had not conducted complete cost 
realism analyses on the interim proposals. 

In this case, the Comptroller General found that 
the agency did not inform Biospherics during discussions 
that it considered its pricing to be unrealistically low. 
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but instead encouraged Biospherics to reduce its pricing to 
a level that the agency ultimately found unrealistic. The 
Comptroller General ruled that the Forest Service had 
conducted inadequate and misleading discussions and 
recommended that it reopen discussions with Biospherics and 
Park.Net, request new final proposal revisions, and 
reevaluate proposals. [Ref. 16] 

b. Mechanical Contractors^ S.A. 

Mechanical Contractors, S.A.; B-277916.2; 
Comptroller General of the United States' 4 March 1998, 
(MCSA) is a protest in which the agency did not conduct 
meaningful discussions. MCSA protested the award of a 
firm, fixed-price contract by the Panama Canal Commission 
(PCC) for cleaning and painting four miter gate leaves in 
the Panama Canal. This protest followed the reevaluation 
of proposals subsequent to a prior sustained protest filed 
by the same protester in regard to the same solicitation. 
The PCC determined that the proposal submitted by Formal 
Management Systems, Inc. represented the best value to the 
Government. 

The RFP specified that offerors were to submit 
technical proposals and price schedules and that technical 
proposals were to be evaluated on the basis of two equally 
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weighted technical factors: 1) technical approach; and 2) 
performance capability. 

The evaluation board in the original source 
selection had evaluated MCSA as excellent or better in 
three technical subfactors and had, therefore, not held 
discussions with MCSA on those particular areas. The 
evaluation board in this case had severely downgraded 
MCSA's scores in the same technical subfactors, but did not 
hold discussions with MCSA regarding the newly discovered 
deficiencies. 

MCSA argued that the PCC should have discussed 
these perceived deficiencies with the company because the 
reevaluation went beyond the scope of the original protest. 
The Comptroller General agreed and found that MCSA was 
prejudiced by the agency's failure to hold discussions on 
the deficiencies identified in the reevaluation. The 
contracting officer specifically cited these deficiencies 
as the basis for his conclusion that the contract schedule 
would be at risk if the contract were awarded to MCSA. GAO 
sustained the protest, but since performance of the 
contract had already begun, its only recommendation was 
that MCSA be reimbursed for its proposal preparation costs 
and the reasonable costs of filing this protest. [Ref. 17] 
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c. Element Swmaary 

In order to perforin fair evaluations with equal 
treatment to all offerors, contracting officers are 
required to perform meaningful discussions in negotiated 
procurements unless the solicitation specifically states 
that award may be made without discussions. Meaningful 
discussions, as defined by the FAR, are those exchanges 
that seek to identify the significant weaknesses, 
deficiencies, or other aspects of the offeror's proposal 
that could be altered or explained to materially enhance 
the proposal's potential for award. [Ref. 2:15.306(d)(3)] 

6. Improper Agency Evaluation of Technical Merit 
The two cases in the following subsections discuss 
protests that were sustained partly due to improper agency 
evaluation of technical merit. This particular sustaining 
element occurred in 17 percent (16.67 percent) of the cases 
analyzed. 

a. ENMAX Corporation 

ENMAX Corporation; B-281965; Comptroller General 
of the United States; 12 May 1999, (ENMAX) is a protest in 
which the agency improperly evaluated a proposal's 
technical merit. ENMAX protested the U.S. Air Force award 
of a fixed-price labor contract to Carlisle Research, Inc. 
(Carlisle) for engineering services at Hill Air Force Base, 
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Utah. The RFP's evaluation scheme stated that award would 


be made using a performance/price trade-off, which 
permitted trade-off between price and the past performance 
evaluation for technically acceptable proposals. Past 
performance and price were equally important. The Air 
Force selected Carlisle as the offeror whose proposal 
presented the best combination of performance risk and 
evaluated price. 

ENMAX alleged that Carlisle's proposal did not 
meet the minimum technical requirements set forth in the 
RFP, and that the Air Force improperly concluded that the 
proposal was technically acceptable. Specifically, the RFP 
stated that proposals shall address each and every area of 
each factor. Since the technical area is rated as 
pass/fail, failure to address even a single factor may 
result in failure of the technical proposal. 

Carlisle's technical proposal did not address 
each and every factor of the technical area, yet the Air 
Force evaluators gave the proposal a rating of "pass" in 
the "demonstrated experience (2 plus years) in 
client/server development in ORACLE, UNIX, NT and 
Powerbuilder environments." 

GAO notes that it is not the internal source 
selection plan, but, rather, the specific language in the 
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RFP that determines whether offerors have been treated 
fairly in an evaluation. In this case, GAO stated that the 
RFP did not specify that failure to meet any one of the 
technical areas would result in a finding of technical 
unacceptability under the factor overall. Further, the RFP 
required offerors to address each and every area within 
each factor, but did not state that failure to do so would 

I 

automatically result in failure of the proposal. So, even 
though the RFP did not mandate failure of this proposal 
given the language of the solicitation itself, GAO 
concluded that it clearly could have affected the 
evaluator's conclusion that Carlisle was technically 
acceptable. Finally, GAO concluded that the evaluators had 
to consider whether Carlisle's failure to demonstrate the 
required experience made it technically unacceptable under 
this evaluation factor. GAO sustained the protest on that 
basis. The Comptroller General recommended that the agency 
reevaluate Carlisle's technical proposal, taking into 
account whether the proposal should be considered 
acceptable overall given Carlisle's failure to show 
experience in the stated computing environments. [Ref. 18] 

To. ATA Defense Industries^ Inc. 

ATA Defense Industries^ Inc.; B-282511, B- 
282511.2; Comptroller General of the United States; 21 July 
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1999, (ATA) is a protest that was sustained because of 
improper technical evaluation by a Federal Agency. ATA 
protested the award of a fixed-price contract by the U.S. 
Army for the Intermediate New Generation Army Targetry 
System. The Army selected Caswell International 
Corporation (Caswell) as the offeror whose proposal 
represented the best value to the agency. 

The RFP identified four evaluation elements: 1) 
technical; 2) performance risk; 3) small business 
utilization; and 4) price. Of these, technical was the 
most important and included the five technical subfactors 
listed below in descending order of importance: 

• Hit detection 

• Overall design and stability of the design 

• Battle effects simulator 

• Radio frequency band width 

• Training/manual 

Of the subfactors, the first two were 
significantly more important than all the others combined. 
ATA's contention was that Caswell's proposal was improperly 
given an excellent rating for hit detection, in violation 
of the Army's evaluation scheme. The Comptroller General 
made extensive inquiries into the technical aspects of the 
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hit detection subfactor and concluded that the Army had 
indeed violated its stated evaluation scheme as delineated 
in the solicitation. The Comptroller General sustained the 
protest and recommended that the Army reopen the 
procurement and reevaluate proposals in accordance with the 
stated evaluation scheme. [Ref. 19] 
c. Element Svcmmaxy 

The cases discussed above represent another 
problem that occurs when Federal contracting officers fail 
to evaluate proposals in a manner consistent with the 
evaluation scheme stated in the solicitation. Whenever an 
evaluation team or source selection authority deviates from 
the selection criteria stated in an RFP, the Government is, 
in essence, inviting a protest. In order to maintain the 
perception of a level playing field for all prospective 
contractors, contracting officers must adhere to the stated 
evaluation scheme and treat all offerors equally. 

7. Improper Agency Evaluation of Labor 
Qualifications 

The following two subsections discuss sustained 
protests in which the agency improperly evaluated the labor 
qualifications of personnel proposed by the prospective 
contractors. In both cases, the protests were sustained 
because the agencies failed to strictly adhere to the RFP's 
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evaluation scheme in assessing the qualifications of an 
offeror's proposed personnel. 

a. Aerospace Design & Fabrication, Inc. 

Aerospace Design & Fabrication, Inc.; B-278896.2, 
B-278896.3, B-278896.4, B-278896.5; 4 May 1998, (ADF) is a 
protest in which improper agency evaluation of labor 
qualifications resulted in sustainment. ADF protested the 
National Aeronautics and Space Administration (NASA) award 
of a contract to Dynacs Engineering Co., Inc. (Dynacs) for 
scientific, engineering, technical and administrative 
services for NASA's Lewis Research Center. The RFP 
anticipated a competitive procurement limited to small, 
disadvantaged businesses and leading to a hybrid contract 
containing approximately ten percent fixed-price and 90 
percent cost reimbursement task orders. The solicitation 
set forth three evaluation factors of equal importance: 1) 
mission suitability; 2) cost; and 3) relevant experience 
and past performance. The relevant subfactor under mission 
suitability was key personnel and was worth one-quarter of 
the total points assigned to the factor. 

ADF argued that Dynacs misrepresented the level 
of commitment from three of its proposed key personnel, 
leading the agency to improperly award Dynacs more credit 
in this evaluation factor than its proposal merited. 
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In reaching its decision, GAO cited previous 
findings of similar circumstances and stated that an 
agreement to work for an offeror, if it wins the 
competition, is nothing more than a promise to negotiate 
for employment and not a binding commitment. GAO went on 
to state that if an offeror falsely states in its proposal 
that certain key personnel have committed to work for them, 
and the agency relies on this misrepresentation in its 
evaluation, then the selection decision is improper. For 
this reason, the Comptroller General found that ADF was 
prejudiced in the award decision and sustained the protest. 
[Ref. 20] 

b. Possebn Consulting^ 

Possehn Consulting; B-278579.2; Comptroller 
General of the United States; 29 July 1998, (Possehn) is a 
protest in which an agency's evaluation ignored evidence of 
required experience, and this resulted in protest 
sustainment. Possehn protested the award of a fixed-price 
contract by the U.S. Forest Service for the location and 
installation of permanent forest inventory plots at various 
national forests. Award was made to Steve Holmes Forestry 
as the offeror whose proposal represented the best value to 
the Government. This protest was the result of a 
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reevaluation of proposals subsequent to a prior sustained 
protest also brought before GAO by Possehn. 

The RFP identified four technical factors in 
descending order of importance: 

• Offeror's experience establishing permanent 
forest and/or vegetation inventory plots 

• Past performance on similar contracts during the 
previous three years. 

• Experience of key management personnel. 

• Qualifications of offered field personnel. 

The relevant factor for purposes of this discussion is the 
final one—qualifications of offered field personnel. The 
agency's record of its evaluation of this factor regarding 
Possehn's proposal incorrectly stated that field crew 
members did not have experience establishing permanent plot 
inventory systems. However, Possehn's final proposal 
revision clearly identified personnel with such experience 
and indicated that they had acquired the experience by 
establishing such plots for the Forest Service. 

In its decision, GAO concluded that the agency 
had improperly overlooked the experience of Possehn's 
proposed field crew and, therefore, it sustained the 
protest. The Comptroller General recommended that the 
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agency reevaluate Possehn's final proposal revision and 
make an appropriate judgment about the value of the 
experience of the personnel offered. [Ref. 21] 

c. Element Summary 

The two cases discussed are examples in which the 
contracting agency improperly evaluated the labor 
qualifications of prospective contractors. In the first 
case, the agency relied on factual misrepresentations in 
the offerors proposal. It may seem reasonable to believe 
that the agency could have done nothing to prevent the 
offeror from misrepresenting itself. However, the agency 
could have significantly reduced the likelihood of relying 
on a misrepresentation had it required signed commitment 
letters from key proposed personnel who were not yet on the 
offeror's payroll. 

In the second case, the agency blatantly erred 
when it reviewed the offeror's final proposal revision, 
overlooking the precise qualifications it desired of field 
personnel. Here, the agency's attention to detail would 
have drastically reduced the possibility of a protest— 
specifically a follow-on protest. 
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8. Failure to Adequately Disclose Factor Evaluation 
Weights in Solicitation 

The single case presented below was the only protest 
among those studied that contained this particular 
sustaining element. The failure to adequately disclose the 
factor evaluation weights in the solicitation was one of 
three sustaining elements noted in the Comptroller 
General's decision for this particular case. 

a. Foundation Health Federal Services^ Inc. 
Foundation Health Federal Services^ Inc; Humana 
Military Healthcare Services, Inc.; B-278189.3, B-278189.4; 
Comptroller General of the United States; 4 February 1998, 
(Foundation/Humana) is a protest in which the agency failed 
to disclose the relative importance of the significant 
evaluation factors. Foundation and Humana protested the 
TRICARE Support Office's (TSO) award of a fixed-price 
contract to Anthem Alliance for Health, Inc. to provide 
health care and associated administrative services for 
Military Health Services System beneficiaries. 

The RFP stated that the agency intended to make 
award to the offeror whose proposal was most advantageous 
to the Government, price and other factors considered. The 
Government intended to award the contract for a base period 
with five one-year options. The weighting ratio of 
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technical area to cost was to be 60 percent/40 percent. 
Within the technical area, the proposal identified seven 
major factors that included 11 technical tasks. Offerors 
were required to respond satisfactorily to all factors and 
subfactors; however, the proposal did not present specific 
weights for each factor and subfactor. 

Foundation and Humana alleged that the 
solicitation did not advise offerors of the evaluation 
criteria TSO actually used and the relative weights it 
assigned to the factors and subfactors. In its discussion, 
GAO referred to Section 2305 of Title 10 of the United 
States Code that requires solicitations to include at a 
minimum "a statement of -(i) all significant factors and 
significant sub factors which the head of the agency 
reasonably expects to consider . . .; [and] (ii) the 

relative importance assigned to each of those factors and 
sub factors ..." GAO reasoned that, where an agency 
fails to advise offerors of the relative weights of the 
evaluation factors, there is no assurance that in selecting 
an offer for award it is obtaining the best value for the 
Government. Finally, the Comptroller General concluded 
that, where the solicitation is silent as to the relative 
importance of the subfactors, offerors can assume that the 
subfactors are weighted approximately equally. In 
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sustaining this protest, the Comptroller General stated 
that TSO improperly failed to adequately disclose the 
weights of the significant evaluation factors and 
subfactors applied in the evaluation. [Ref. 22] 

b. Element Svmmary 

The preceding discussion represents the single 
instance of an agency's failure to adequately disclose 
factor evaluation weights in a solicitation for the period 
studied. This does not diminish its importance, however. 
As GAO pointed out, without factor evaluation weights 
specified in the solicitation, there is no assurance that 
the Government is obtaining the best value for its money. 

D. INTEGRATED ANALYSIS 

The purpose of this analysis is to provide Federal 
contracting officers with the recommendations designed to 
address the underlying causes of sustained protests 
discussed above. 

1. Adequately Support the Selection 

Decision with Reasoned Documentation 

In case after case, whether or not it is cited as a 
sustaining element in the decision, GAO notes that 
personnel are failing to adequately document their 
decisions. This happens at all levels within the source 
selection team. Yet, this particular deficiency is. 


85 



intuitively, the easiest to correct, particularly since GAO 
presumes the actions of agencies correct unless proven 
otherwise. In this respect, agencies are given great 
leeway in their decision. However, with this latitude 
comes the responsibility to demonstrate that all decisions 
are reasonable and not arbitrary. 

The primary way to ensure proper documentation is 
through adequate training. By ensuring that personnel 
understand the importance of documenting the basis of 
business decisions, contracting agencies can reduce the 
risk of sustainable protests. Training can be incorporated 
into the source selection plan or conducted on a just-in- 
time, as-needed basis. Such training should stress not 
only the importance of documentation, but also the tools 
necessary to accomplish the task. These tools might 
include word processing training as a foundation upon which 
to build and can be tailored to the individual demands of 
the agency. 

Another way to accomplish adequate documentation might 
be the use of standardized evaluation forms by all 
personnel involved with evaluating proposals. In addition, 
a specific form could be developed for personnel who may 
have to make trade-off decisions. Such a form could 
specify the aspect of the proposal being considered for a 
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trade-off, the benefits or risk expected, and any impact on 
risk that may result. 

Adequate staffing could also reduce the risk of a 
sustained protest due to inadequate supporting 
documentation. In the current atmosphere of fiscal 
restraint, agencies may have insufficient funds for 
training personnel properly. Drawing upon personnel who 
already possess sufficient skills in evaluating and 
documenting source selections is one means of overcoming 
such financial obstacles. Unfortunately, smaller 
organizations may not have the luxury of a large pool of 
personnel from which to draw experienced team members. 

Next, using management more aggressively can help 
reduce the risk of a sustainable protest. By carefully 
reviewing the documentation of source selection team 
members, managers can verify that the supporting rationale 
for decisions is both present and appropriate. Admittedly, 
this requires an investment of the manager's time, but as 
the cases analyzed in this thesis have shown, it would be a 
worthwhile investment. 

Biospherics demonstrated that if it were possible to 
rely solely on the documents prepared in response to 
protest contentions, an agency's selection decision might 
seldom be found inadequate. However, the Comptroller 
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General has repeatedly stated that, although the entire 
record will be considered in protest resolutions, greater 
weight will be given to the contemporaneous source 
selection materials. If there had been any notation to 
support the relative strengths and weaknesses that 
evaluators observed in the reviewed proposals, the case 
would undoubtedly have turned out differently. 

f 

[Ref.5:p.739] 

Finally, the controlling element of the source 
selection process requires adequate supporting 
documentation. FAR part 15.608 (a) (3) requires 
documentation to support the evaluation of proposals, 
includingI the basis for evaluation and analysis of the 
technically acceptable and unacceptable proposals; an 
assessment of each offeror's ability to accomplish the 
technical requirement; and a summary of findings. The FAR 
goes on to state that the basis for the source selection 
decision and the relative differences among proposals must 
be documented. In this selection decision documentation, an 
agency must describe the strengths, weaknesses and risks of 
the various proposals in terms of stated evaluation 
criteria. [Ref. 2:15,608(a)(3)] 
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2. Evaluate Proposals in a Manner Consistent with 
the Solicitation 

Prospective contractors have a right to expect that 
their proposals will be evaluated in accordance with the 
stated evaluation scheme of the solicitation. Unless the 
solicitation has been amended, the entire evaluation must 
be based on the stated factors throughout the competitive 
negotiation process. [Ref.5: p.543] 

Contracting agencies can ensure that proposals are 
evaluated consistently in a number of ways. First, the 
source selection plan describes the evaluation factors and 
subfactors, their relative importance, and the methodology 
used to evaluate proposals. This plan also serves as a 
guide for the entire source selection team on the roles of 
the members and the conduct of the source selection from 
proposal evaluation, through the cost/technical trade-off, 
to award decision and debriefing. [Ref.5: p.210] 

In Marquette, the solicitation invited offers to 
either replace or upgrade existing equipment. The 
solicitation did not provide for any method of 
"normalizing" the two distinct approaches. Yet, this is 
exactly what the source selection authority did as part of 
the price analysis it conducted. Since this procedure for 
comparing proposals was not specified in the solicitation. 


89 



GAO decided in favor of the protestor. By specifying in 
the solicitation the procedure for comparing the two 
different approaches, the source selection authority could 
reasonably have applied this normalization technique and 
significantly mitigated the risk of a sustained protest. 

Second, it is essential that selection personnel have 
both experience in evaluating proposals and technical 
expertise in the area they are assigned to evaluate. 

Again, in Marquette, not only did the source selection 
authority apply a comparative technique not specified in 
the solicitation, but it also applied the technique 
incorrectly. The official apparently lacked the technical 
expertise to recognize the double-counting mistake made. 

The next risk-reduction approach for source selection 
teams is in the area of management. As discussed in the 
previous recommendation, by effectively directing the 
members of the source selection team in properly evaluating 
proposals and then verifying the results, senior personnel 
can help ensure consistent evaluations. 

Finally, there are statutes and regulations dictating 
that evaluations shall be based solely on the factors 
specified in the solicitation. By adhering to these 
principles, contracting agencies can further mitigate the 
risk of sustainable protests. [Ref.5:p.542] 



3. Thoroughly Conduct Cost/Price Realism Analyses 
when Required, and Always Evaluate the Total 
Cost/Price 

Improper agency evaluation of cost/price was a factor 
in nearly one-third of all sustained protests for the 
period studied. In Tidewater, the Navy's cost realism 
evaluation consisted solely of comparing each offeror's 
target cost with the Government estimate. Since this was a 
cost-reimbursement contract, a cost realism analysis was 
required to determine if the overall costs were realistic 
for the work to be performed and reflected a clear 
understanding of the agency's requirements. In the 
planning stage of the acquisition, the agency could have 
considered the risk associated with not conducting a cost 
realism analysis and made allowances in its source 
selection plan. Specifically, the agency could have 
engaged the services of the Defense Contract Audit Agency 
(DCAA) to perform a pre-award survey. Such forethought by 
the source selection team involved in Hamm could have 
averted that unfavorable decision. In that case, the 
contracting officer accepted a proposed overhead rate that 
was significantly lower than DCAA's approved rate. This 
fatal flaw could have been avoided if the agency had 
planned to have DCAA evaluate proposed overhead rates. 
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Effective market research can also aid in determining 
price realism. By identifying adeguate competition through 
market research and promulgating the solicitation to 
numerous qualified and capable contractors, the FAR has 
stated that such competition will usually establish price 
reasonableness. [Ref. 2:15.305] 

4. Equitably and Consistently Evaluate Past 
Performance 

Agencies improperly or unfairly evaluated a 
contractor's past performance in 20 percent of the 
sustained protests analyzed. More importantly, it is one 
of the most significant evaluation factors because it is a 
mandatory consideration on all procurements over $1 
million. [Ref.5: p.594] 

U.S. Property is a classic example of unfairly 
evaluating a contractor's past performance. In essence, 
two newly formed companies—^whose presidents had both 
previously worked for the incumbent contractor—each claimed 
virtually the same experience in their respective 
proposals. One was evaluated as having acceptable prior 
experience, while the other's experience was rated as only 
marginal. As in the last section, more thorough market 
research may have eliminated the risk of a sustained 
protest. If the past performance stated in each proposal 


92 



had not obviously shown that both companies' senior 
leadership shared the same prior work experience, then 
effective market research might have brought this 
information into plainer view. Comprehensive discussions 
on this subject should have made the inequity apparent to 
evaluators. Instead of relying on evaluators to sift 
through reams of proposal paperwork to find the pertinent 
information, adequately structured and fairly conducted 
discussions could have assisted in leveling the playing 
field as far as past performance was concerned. 

In Nova, the company's actual past performance had 
been rated "average to outstanding" by previous contracting 
agencies. Evaluators who, instead, focused on the fact 
that Nova had numerous unresolved contract claims 
apparently overlooked this. Had these same evaluators 
delved deeper into the issue, they would likely have 
reached the same conclusions as GAO. After examining the 
claims closely, there was no evidence that these claims 
indicated problems in management effectiveness, 
responsiveness or cooperation. [Ref.13]. 

Finally, personnel must be instructed on how to 
evaluate a lack of past performance. The FAR Part 15 
states that, in the case of an offeror without a record of 
relevant past performance or when past performance 
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information is not available, the offeror may not be 
evaluated favorably or unfavorably on past performance 
[Ref. 2:15.305(a)(2)(iv)] 

This presents both a training challenge and a 
management concern. The challenge is to teach source 
selection team members about the regulation, as well as how 
to apply the regulation when evaluating proposals. The 
management concern is attempting to ensure that an 
offeror's lack of past performance information does not 
subliminally taint the evaluator's proposal rating. It is 
one thing to train the individual regarding the 
regulations, but it is another matter entirely when an 
individual's perceptions creep into the evaluation of other 
factors, such as technical ability to perform the contract. 
Only through strict adherence to the specific evaluation 
criteria can managers ensure that source selection 
personnel have not inadvertently prejudiced a prospective 
contractor. 

5. Conduct Meaningful Discussions that Draw Offerors 
into Areas of their Proposal that Require 
Amplification or Correction 

By failing to conduct meaningful discussions with 
offerors, contracting agencies were responsible for just 
over 20 percent of all sustained protests in calendar years 
1998 and 1999. The purpose of discussions is to point 
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offerors toward areas of their proposals that are unclear 
or in error, and to then allow them to revise their offer. 
It can also be a signal that the solicitation itself did 
not clearly describe the agency's requirement. If that is 
the case, contracting officers should cancel the 
solicitation, revise it and then resolicit the requirement, 
or amend the current solicitation so that all offerors are 
proposing based on the same requirements. Aside from the 
requirement to hold meaningful discussions with offerors in 
the competitive range, there is also a requirement to give 
offerors an opportunity to address any adverse past 
performance information on which they have not previously 
had an opportunity to comment. 

Biospherics goes one step beyond failing to conduct 
meaningful discussions. In this case, the agency suggested 
that the contractor review its proposal for any possible 
further savings when, in fact, the agency ultimately 
decided the offeror's price was unrealistically low. Such 
misleading communications undermine the contractor's 
confidence in Government as a fair business partner and 
further promote the "Us versus Them" mentality that 
contracting agencies must constantly battle. This is 
another element of the source selection process that can be 
improved trough training. Effective training can not only 
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increase the knowledge level of source selection team 
members, but also improve their interpersonal communication 
skills. Both of these benefits to training will improve 
the value of contract discussions. 

Mechanical is a protest that follows the reevaluation 
of proposals as the result of a prior sustained protest 
involving the same solicitation. GAO sustained the initial 
protest noting that the agency had not adequately supported 
it's proposal evaluations and because the evaluations 
lacked reasonable bases. The evaluation board in the 
original source selection had evaluated Mechanical as 
excellent or better in three technical sub factors and had 
therefore not held discussions regarding those sub factors. 
In the follow on review, the evaluation board severely 
downgraded the offeror in those same sub factors but did 
not hold discussions regarding the newly discovered 
deficiencies. GAO found that since these deficiencies had 
not been addressed in previous discussions they could not 
form the basis for the contracting officer's decision to 
eliminate the offeror from consideration. The lesson that 
contracting agencies should derive from Mechanical is that 
discussions must be tailored to the specifics of the 
solicitation and the unique offer in order to be 
meaningful. As mentioned previously, meaningful 
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discussions 1) should lead offerors to areas of their 
proposal that require amplification or correction, 2) must 
address adverse past performance information which the 
offeror has previously not had the opportunity to comment 
on, and 3) should seek to level the playing field of all 
offerors without resorting to technical leveling or 
technical transfusion. 

6. Craft a Con^lete Solicitation that Addresses the 
Agency's True Requirements and Clearly Instructs 
Offerors in Proposal Preparation cUid Evaluation 

Requirements generation and acquisition management 
represent two of the three key management processes, 
discussed in DoD Directive 5000.1, 15 March 1996, that must 
dovetail effectively to ensure success. The third process 
is the planning, programming and budgeting system. This 
philosophy is reflected in the FAR which states that a 
request for proposal must contain at a minimum 1) 
Government's requirements, 2) anticipated terms and 
conditions that will apply to the contract, 3) information 
required to be in the offers, and 4) factors and sub 
factors that will be used to evaluate the proposal and 
their relative importance. [Ref. 2:15.203(a)] 
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Once again the primary deterrent of sustained protests 
is an investment of time. By applying significant effort 
in the planning stage of an acquisition, contracting 
agencies can help ensure that contract performance is on 
time, within budget and meets the needs of the end user. 

Contracting agencies are afforded broad discretion in 
the evaluation process. Provided that proposals are 
evaluated reasonably and that award is made in accordance 
with the evaluation scheme described in the RFP, GAO will 
decide for the Government. 

Another planning tool contracting agencies can use in 
an effort to reduce the risk of a sustained protest is to 
involve prospective contractors early in the process. 

Early supplier involvement helps ensure that the user's 
requirements 1) are technically capable of being met, 2) 
can be met within a specified timeframe, and 3) can be 
accomplished at an affordable price. Involving industry at 
the start of the acquisition process also allows for 
innovative technologies to be proposed and helps foster a 
sound business relationship for future efforts. 

E. CONCLUSION 

This chapter provided a qualitative analysis of the 42 
cases of sustained protests of contracts awarded using a 
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trade-off process. First, the chapter presented statistics 
to provide a better understanding of the types and 
frequency of sustained elements observed by the researcher. 

The eight sustaining elements observed among the cases 
studied were, in descending order of frequency: 

• Selection Decision not Adequately Documented 

• Evaluation of Proposals Inconsistent with RFP 

• Improper Agency Evaluation of Cost/Price 

• Improper Agency Evaluation of Past Performance 

• Agency Improperly Failed to Conduct Meaningful 
Discussions 

• Improper Agency Evaluation of Technical Merit 

• Improper Agency Evaluation of Labor 
Qualifications 

• Failure to Adequately Disclose Factor Evaluation 
Weights in Solicitation 

In the researcher's opinion, these eight common 
sustaining elements can be further reduced to three guiding 
principles that Federal agency contracting officers can 
follow in order to mitigate the risk of sustainable 
protests. They are: 

• Treat each offeror fairly, ethically and 
equitably, both in relation to other offerors and 
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as a prospective contractor seeking to engage in 
business dealings with the Government. 

• Evaluate each and every proposal strictly in 
accordance with the stated evaluation scheme set 
forth in the solicitation. 

• Thoroughly evaluate the full merits of every 
proposal that is submitted, and completely 
document, with reasoned explanation, every 
business decision made on behalf of the 
Government. 

Clearly it is not possible to completely avoid 
protests by prospective contractors, but through reasoned 
and even-handed practices, it is possible to mitigate the 
risk of sustainable protests. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

This chapter first provides answers to the primary and 
secondary research questions, which were the foundation of 
this thesis. It then presents conclusions and 
recommendations drawn from the qualitative analysis in 
Chapter IV. Finally, the chapter recommendsed areas for 
further research regarding protests of contracts awarded 
using a trade-off process. 

B. RESEARCH QUESTIONS 

In order to accomplish the objectives of this thesis, 
fundamental research questions were formulated. The 
responses to the primary and secondary research questions 
are provided in this section. 

1. Primary Research Question 

To what extent will an analysis of General Accounting 
Office decisions of protests of contracts awarded using the 
best value trade-off process^ provide insight into the 
nature of Government contract protests and suggest ways to 
reduce such protests in the future? 

Analyzing sustained protests of contracts awarded 
using a best value trade-off process provided a significant 
understanding of the basis for GAO's decisions. The 
following are selected general observations: 
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• Protests were sustained most frequently because 
of an agency's failure to evaluate proposals in 
accordance with the evaluation scheme and because 
business trade-offs made in the selection 
decision were not adequately supported by 
reasoned documentation. 

• An agency's inability to properly evaluate all 
aspects of an offeror's proposal-cost/price, past 
performance, technical merit and labor 

qualifications-was a common fatal flaw among 
sustained protests. 

• The fact that a Federal agency elects to use a 
trade-off source selection process does not 
increase the sustainment rate of protests 
compared to all protests of contracts awarded. 

• Many protests reflect ineffective communications 
between offerors and contracting agencies during 
the evaluation process. 

The following are suggestions for mitigating the risk 
of sustained protests of contracts awarded using the trade¬ 
off process: 

• Document all business decisions and trade-offs 
made in evaluating offerors' proposals. 

• Evaluate proposals strictly in accordance with 
the evaluation scheme stated in the 
solicitations. 

• Conduct thorough and adequate evaluations of all 
aspects of an offeror's proposal. 

• Conduct meaningful discussions. 

• Structure solicitations to adequately reflect the 
factor evaluation weights to be applied in 
proposal evaluation. 
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2. Secondary Research Questions 

a. Vfbat will an analysis of Govenment contract 
protests from January 1998 through December 
1999 suggest about the nature of Goveinment 
contract protests? 

Based on an analysis of 325 protests of contracts 
awarded using the trade-off process, the researcher 
observed the following general trends: 

• Government tends to win seven out of every 
eight protests. 

• The number of protests filed against DoD 
awards was nearly two-fifths greater than 
for all other Federal agencies combined. 

• The sustainment rate was slightly lower for 
protests of DoD awards compared to all other 
Federal agencies as a group. 

• One in eight protests had previously been 
brought before the Comptroller General 
(i.e., same contract award). 

• The most common sustaining element was 
failure to adequately document the source 
selection decision. 

• Defense Finance Accounting Service had the 
highest sustainment rate among DoD 
activities, with 67 percent. 

• The Department of Health and Human Services 
had a sustainment rate of 40 percent, the 
highest among other Federal agencies. 

• DoD accounted for exactly half of the 
sustained protests. 

b. Do sustained protests of awards using the 
best value trade-off process fall Into 
distinct categories? 
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The results of the researcher's analysis show 
that sustained protests of contracts awarded using the 
trade-off process fall into one or more of the following 
categories: 

• Selection Decision Not Adequately Documented 

• Evaluation of Proposals Inconsistent with RFP 

• Improper Agency Evaluation of Cost/Price 

• Improper Agency Evaluation of Past Performance 

• Agency Improperly Failed to Conduct Meaningful 
Discussions 

• Improper Agency Evaluation of Technical Merit 

• Improper Agency Evaluation of Labor 
Qualifications 

• Failure to Adequately Disclose Factor Evaluation 
Weights in Solicitation 

c. What will an analysis of the research results 
suggest about weaknesses in Govemr^nt 
contracting norms and execution practices? 

The research revealed eight categories of 

sustaining elements in which the Government exhibited 

weaknesses in its source selection norms and execution 

practices. An analysis of these elements indicates that 

the primary weakness is the failure of those involved in 

the source selection process to adequately support the 

trade-off decision with reasoned documentation. The second 

most frequent weakness observed also involved various 
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members of the source selection process. Failure to adhere 

to the evaluation scheme set forth in the solicitation 

constituted one-third of all source selection weaknesses. 

The research indicated that there is a need for on-going 

training among all members of the source selection team and 

that the focus of such training should be in the proper 

methods for proposal evaluation, conduct of discussions and 

source selection documentation. 

d. How can the results of this analysis be used 
to reduce the risk of protest? 

There is very little that Federal contracting 

agencies can do to reduce the risk of a contract protest. 

After all, prospective contractors have little to lose; for 

the price of a stamp, they can file a protest with the 

Comptroller General. Frivolous protests, however, will be 

summarily dismissed. This leaves the question: "How can 

the number of legitimate protests be reduced?" The answer 

can be found in the following two steps: 

• Ensure the integrity of the source selection 
process, as described in Chapter II. 

• Adequately inform unsuccessful offerors, during 
the debriefing process, of the reasons their 
proposal was not selected. 

By following these steps. Federal contracting 
agencies can reduce the risk of protest. 
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e. How can the results of this analysis be used to 
reduce the risk of sustainable protest? 

This research revealed that, with one exception, 

the basis for sustainment occurred during the evaluation of 

proposals and resulted in the Comptroller General ruling 

that a particular aspect of the source selection process 

was unreasonable, improper or unsupported. In order to 

effectively mitigate the risk of a sustainable protest, 

contracting agencies must first ensure that proposals are 

properly evaluated in accordance with governing regulations 

and the specifics of the solicitation. Second, contracting 

agencies must adequately document the rationale for their 

trade-off decisions. Performing this second step not only 

helps protect against successful protests by demonstrating 

that the decision is neither arbitrary nor capricious, but 

also provides a framework on which to base the debriefing 

of unsuccessful offerors. By thoroughly understanding the 

reason for the agency's trade-off decision, unsuccessful 

offerors begin to perceive the selection process as 

equitable and logical. 

C. CONCLUSIONS 

The answers to the research questions have led the 
researcher the following conclusions: 

• Prospective contractors' protests are usually 
denied unless they can show that a selection 
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decision is unreasonable or conflicts with the 
stated evaluation criteria for award. 

• Contracting agencies would benefit from training 
in the areas of source selection and decision 
documentation. 

• Communication between offerors and the Government 
appears to be problematic with respect to 
discussions. 

• Contracts of all types and sizes are subject to 
considerable delay due to protests. This was 
reflected in the five cases that had been brought 
before the Comptroller General a second time and 
that resulted in an average period of 19 months 
from release of RFP until ultimate sustainment by 
GAO. This does not even indicate the extent of 
the delay until actual contract performance 
begins. 

D. RECOMMENDATIONS 

This research suggests the following recommendations 
for mitigating the risk of sustainable protests: 

• Adequately support the selection decision with 
reasoned documentation. 

• Evaluate proposals in a manner consistent with the 
solicitation. 

• Thoroughly conduct cost/price realism analyses when 
required, and always evaluate the total cost/price. 

• Equitably and consistently evaluate past performance. 

• Conduct meaningful discussions that draw offerors into 
areas of their proposal that require amplification or 
correction. 

E. AREAS FOR FURTHER RESEARCH 

The scope of this research was limited to protests of 
contracts awarded using the trade-off process for calendar 
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years 1998 and 1999, with attention focused on protests 
that were sustained and the reasons for sustainment. 
Specific areas that merit further research include: 

• Comparison of GAO decisions with those of the U.S. 
Court of Federal Claims. 

• Analysis of U.S. Court of Federal Claims decisions for 
a similar period. 

• Analysis of those contracting agencies with the 
highest protest sustainment rates, focusing on why 
these agencies have higher sustainment rates than 
other agencies. 

• Continuation of this thesis, focusing on types of 
products or services being solicited. 

• Development of a center of excellence training program 
for training source selection teams that will be 
awarding contracts using the trade-off process. 

• A study determining if early supplier involvement and 
teaming with industry practices have affected the 
level of protests of contracts awarded using the 
trade-off process. 
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APPENDIX A. SELECTED ACRONYMS 


Army 

BAFO 

CO 

COFD 

CPAF 

CPFF 

DCA 

DFARS 

DFAS 

DISA 

DLA 

DoA 

DoC 

DoD 

DoE 

DoJ 

Don 


CoE Army, Corps of Engineers 

Best and Final Offer (see FPR) 

Contracting Officer 

Contracting Officer's Final Decision 

Cost-Plus-Award-Fee 

Cost-Plus-Fixed-Fee 

Defense Commissary Agency 

Defense Federal Acquisition Regulation 

Supplement 

Defense Finance and Accounting Service 

Defense Information Systems Agency 

Defense Logistics Agency 

Department of Agriculture 

Department of Commerce 

Department of Defense 

Department of Energy 

Department of Justice 

Department of Navy 
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Dot 

DSCP 

DSCR 

EPA 

FAR 

FFP 

FHA 

FPAF 

FPIF 

GAO 

GSA 

HHS 

HUD 

IDIQ 

JFK-CPA 

LPTA 


Department of Transportation 
Defense Supply Center Philidelphia 
Defense Supply Center Richmond 

Environmental Protection Agency 

Federal Acquisition Regulation 
Firm Fixed-Price 
Federal Highway Administration 
Fixed-Price Award Fee 
Fixed-Price-Incentive-Firm 

General Accounting Office 
General Services Administration 

Health & Human Services (Department of) 
Housing & Urban Development (Department of) 

Indefinite Delivery, Indefinite Quantity 

J.F. Kennedy, Center for the Performing Arts 

Lowest Priced, Technically Acceptable 

no 



NASA 

NIEHS 

PCC 

f 

RFP 

RFQ 

SSA 

SSAC 

SSEB 

SSP 

SS Admin 

TSO 

USAID 

USMS 

uses 

VA 


National Aeronautical & Space Administration 
National Institute of Environmental Health 
Service 

Panama Canal Commission 

Request for Proposal 
Request for Quotation 

Source Selection Authority 
Source Selection Advisory Council 
Source Selection Evaluation Board 
Source Selection Plan 
Social Security Administration 

TRICARE Support Office 

U.S. Agency for International Development 
U.S. Marshal Service 
U.S. Geological Survey 

Veteran's Affairs (Department of) 
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APPENDIX B. AMC PAMPHLET 715-3 


AMC Pamphlet 715-3 

Contracting for Best Value 
A Best Practices Guide to Source Selection 

1 January 1998 


Department of the Army 

Headquarters, United States Army Materiel Command 
5001 Eisenhower Avenue, Alexandria, VA 22333-0001 

Foreword 


This guide provides techniques and practices for obtaining best value products and services 
through source selection. Consistent with the spirit of acquisition reform, it introduces new and 
inn ovative techniques to simplify the source selection process and produce better value. Its 
purpose is to provide you with a practical reference tool that will help you implement a new way 
of doing business that promotes flexibility, streamlining, and simplified procedures. 

This guide is designed for use by the entire acquisition workforce to promote a consistent 
imderstanding of best value and the various processes and techniques that can be used to achieve 
it. It explains best practices for planning your source selection, teaming, exchanging information 
with industry, and conducting efficient and effective source selections. 

I encourage you to read and use this guide in your efforts to get the best value for your 
customers. 


Johnnie E. Wilson 
General, USA 
Commanding 

U.S. Army Materiel Command 

Acknowledgments 

We are issuing this guide to help the acquisition community and its customers conduct efficient 
and effective source selections that produce better value. It should be used in conjunction with 
existing regulations. 

This guide was a team effort. Shelley Scott, Jim Bozzard, Ann Budd, and Sally George 
consolidated, revised, and updated the guide to reflect the latest acquisition reform initiatives. 
We thank Diane Travers of the AMC Office of Command Counsel for her assistance and special 


113 






contributions. We also extend our thanks to Curtis Stevenson from the Army staff for his 
support and advice. Finally, we acknowledge and thank all of the Source Selection Improvement 
Integrated Product Team members and the individuals within their networks across the Army 
who reviewed the drafts, contributed invaluable comments and recommendations, and so 
graciously shared their knowledge and experience. 

We encourage you to share this guidance with those who may benefit. 

We are distributing only a limited number of hard copies since we have made the guide 
electronically accessible as follows: 

You may view, download and provide feedback on this guide via the Internet through the AMC 
Source Selection Resource Center at http://www.army-acquisition.net/amcweb/fr_ssll.htm. 

This guide is also included within the DOD Acquisition Deskbook, accessible via the Internet at 
http://web.deskbook.osd.mil or on CD-ROM. 

The Source Selection referenced throughout the guide are designed to be living 

models for commonly used source selection documents. While they are not available with this 
hard copy, they may be accessed electronically*. 


This pamphlet does not create any substantive or procedmal right in 
thir d parties or impose any specific legal duty or obligation upon any 
government organization or employee. 


This guide consolidates and updates all AMC source selection guides into a single document that 
includes separate appendices on past performance, oral presentations, and debriefings. It 
should be used to promote best value in source selection. 

* This pamphlet supersedes AMC-P 715-3 VoL 1,8 Jan 87; AMC-P 715-3 VoL 

2, 9 Oct 87; AMC-P 715-3 VoL 3,18 Mar 87; AMC-P 715-3 VoL 4,15 Sep 
93; AMC-P 715-3 VoL 5,16 Aug 94; and AMC-P 715-3 VoL 6,24 Jan 95. 
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Overview 


Source selection is the process used in competitive, negotiated contracting to select the proposal 
expected to result in the best value to the Government. The source selection approach must be 
tailored to the acquisition. You have to consider your evalxiation needs; i.e., don’t make source 
selection more complicated and expensive than necessary. 

What is Best Value? 

In the broadest sense, best value is the outcome of any acquisition that ensures we meet the 
customer’s needs in the most effective, economical, and timely manner. It’s the result of the 
unique circumstances of each acquisition, the acquisition strategy, choice of contracting method, 
and award decision. Under this concept, best value is the goal of sealed bidding, simplified 
acquisition, commercial item acquisition, negotiated acquisition, and any other specialized 
acquisition methods or combination of methods you choose to use. 


Best Value is the goal 
of every acquisition 


Purpose of This Guide 

T his guide provides information on the various processes and techmques that can be used to 
conduct efficient and effective source selections. We are presenting some of the best and most 
innovative practices being used. The principles in this guide apply to all source selections, both 
those that are complex and the majority where the contracting officer is the selection official. 
During acquisition planning, select the methodology that is most appropriate to the umque 
circums tan ces of the acquisition and expected to result in the best value. 

Unless you use a lowest price techniceilly acceptable evaluation approach, your source selection 
will involve some form of tradeoff. This guide’s focus is on the tradeoff process and will 
provide some hints and ideas that will be useful in doing a tradeoff between cost or price and 
other important factors. 

There are two important points to keep in mind as you do your planning and select your 
evaluation and source selection process: 

• Tailor your process to fit your circumstances. There is no magic checklist in this 
arena. Consider the complexity of the acquisition and resources available. Use a 
combination of techniques if it will work best for you and if it is fair. 

• The same principles apply in selecting and executing a source selection process or 
technique, whether you are using a formally structured organization for a complex 
acquisition or a more streamlined process typical for the majority of source 
selections. 
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The Federal Acquisition Regulation (FAR) prescribes the general policies governing source 
selection. 

Appendix A of this guide contains definitions of certain words and terms associated with source 
selection. 

Appendix B contains references along with their Internet addresses to assist you further in 
contracting for best value and conducting a somrce selection. 

The Source Selection "©R/iiqpww^s” referenced throughout this guide are models for commonly 
used source selection dociunents and procedures and should be tailored to each individual 
acquisition. They may be accessed only on electronic versions, e.g., http://www.army- 
acquisition.net/amcweb/fr_ssll.htm. 

Importance of the Source Selection Authority 

The consequences of the selection decision can be far-reaching. In most cases the contracting 
officer is the selection official. In some acquisitions, or class of acquisitions, the agency head or 
other official may be the selection official, or will appoint someone else to make the selection. 
The source selection authority must be at a level that is fully accoxmtable for the results of the 
decision and knowledgeable of the factors necessary to determine the best value. In addition, 
successful execution of an acquisition using the tradeoff process requires early involvement of 
the source selection authority so that person is prepared to make a rational selection decision 
consistent with the solicitation. The amount of time and effort required obviously needs to be 
considered when making the appointment. 

In a complex source selection, it may be useful to provide a number of briefings to the somce 
selection authority early in the acquisition process and at critical steps throughout the process. 
This approach will ensure that the source selection authority knows the program and the 
acquisition process constraints. It also allows the source selection authority to readily express 
concerns and ideas that are hkely to influence the final selection decision. 

Examples of where source selection authority involvement is essential include approval of the 
source selection/evaluation plan and the solicitation. 

Importance of Procurement Integrity 

There are stringent requirements for maintaining the integrity of the procurement process that 
Must be adhered to by all participants involved in the somce selection process. This includes 
both technical and contracting personnel. Procurement integrity rules provide for both civil and 
criminal penalties for violations (see FAR 3.104). The guiding principle behind these 
requirements is that all offerors are treated fairly and no one obtains an unfair advantage. 


116 



Planning For Source Selection 

Designing an Acquisition Strategy 

As soon as possible after a need to acquire products or services has been identified, an 
acquisition strategy meeting should be held. The attendees should include the person responsible 
for managing the program or project, acquisition and legal representatives, potential evaluation 
team members, and others as needed. 

The strategy meeting should be used to determine the acquisition approach including the source 
selection process and techniques that will be most appropriate. The group should use the 
meeting to discuss the results of market research, potential evaluation factors, information that 
may be needed from offerors to support those factors, and other appropriate planning issues such 
as tibe timetables for the acquisition and who should be members of the evaluation team. The 
group should design a strategy that best reflects the specific requirement, the results of market 
research, and the risks associated with the acquisition. The information obtained in the strategy 
meeting will be used as a basis for developing the source selection/evaluation plan. 


Source selection 
is a team effort 


Forming a Team 
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Source Selection “Blueprint” 
Typical Complex Source Selection Team 



Blueprint; Typical Complex Team 
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Source Selection “Blueprint” 

Typical Non-Complex Source Selection Team 



SOURCE SELECTION AUTHORITY (SSA) 
(usually the contracting officer) 
COMPARES AND DECIDES 


EVALUATOR 
Evaluates Cost 



EVALUATOR 
Evaluates Technical Merit 
and Proposal Risk 



EVALUATOR 
Evaluates 
Performance Risk 


Evaluators evaluate proposals against 
RFP factors and subfactors. Assist the 
SSA in comparative analysis, if requested 


Biiieprint; Typical Non-Complex Team 

Source selection should be a multidisciplined team effort from the earliest planning stages. 

The size and composition of the team should be tailored specifically to the acquisition. In 
complex source selections you may have a larger team (e.g., 8 to 10 people) from various 
functional disciplines. In streamlined source selections, however, the team may consist of one or 
more technical evaluators and the contracting officer, who is also the source selection authority. 
Whether the team is large or small, it should be established to ensure continuity and active 
ongoing involvement of appropriate contracting, technical, logistics, legal, user, contract 
administrators, and other experts to ensure a comprehensive evaluation of each proposal. 

Researching the Market 

Market research is the first step in any acquisition and an essential part of designing every 
acquisition strategy. The acquisition team uses market research to obtain information on 
products and services available in the commercial marketplace. Market research is key in 
determining whether a need can be met by a commercial item or nondevelopmental item and in 
identifying commercial practices associated with such items or services. It also has a key impact 
on your choice of appropriate evaluation factors, contracting method, and the amount and type of 
information to be included in proposals. 
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A thorough research of the market should be done as soon as needs are forecast and as part of 
acquisition planning. Sometimes it might be a one-person effort. Other times a team effort. A 
variety of techniques may be used to conduct market research and may include: 

• Contacting knowledgeable individuals regarding market capabilities; 

• Reviewing the results of recent market research; 

• Querying government or commercial data bases; 

• Participating in interactive, on-line coimmmication; 

• Reviewing catalogs and product literature. 

Determining the Source Selection Approach 

One of the first steps in designing an acquisition strategy is to determine the source selection 
approach or combination of approaches that you will use to obtain the best value. At either end 
of the best value continuum, are the tradeoff process and the lowest price technically acceptable 
process. 

Other source selection processes can be designed to fit particular circumstances. You could 
tailor the process to combine elements of these two approaches. You could also use oral 
presentations as part of the proposal submission. The point is that the source selection processes 
or techniques must be appropriate to the acquisition. 

The Tradeoff Process 

Cost or price is always an evaluation factor in any source selection. However, many times you 
may have other factors that you also want to consider. You may need techmcal capabilities, 
qualifications, or experience that a low cost/price offeror may not possess. These factors may or 
may not be more important than cost/price, but they do have a strong bearing on the source 
selection decision. The source selection authority needs flexibility to select the best value that 
may not be the lowest price or the highest technically rated offeror. The decision will involve a 
comparison of the combination of noncost strengths, weaknesses, and risks and cost/price offered 
in each proposal and judgment as to which provides the best combination. The source selection 
authority will have to document the decision and why the selected source represents the best 
value to the government. This is the essence of the tradeoff process. 

When to Use the Tradeoff Process; Strengths and Potential Pitfalls 

Use the tradeoff process when it is essential to evaluate and compare 
factors in addition to cost or price in order to select the most 
advantageous proposal and obtain the best value. 

The tradeoff process is particularly appropriate if: 

• The Government’s requirements are difficult to define, complex, or historically 
troublesome; 
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• You expect measurable differences in the design, performance, quality, reliability, or 
supportability; 

• Services are not clearly defined or highly skilled personnel are required; 

• You are willing to pay extra for capability, skills, reduced risk, or other noncost 
factors, if the added benefits are worth the premium; 

Always consider the strengths and potential pitfalls of using a tradeoff process to ensure that it is 

consistent with yom overall acquisition strategy. 

Strengths 

• Allows greater flexibility to subjectively compare technical and cost factors to 
determine the value of the relative strengths, weaknesses, and risks of the proposals. 

• Enables selection of the best approach among a range of solutions and increases the 
likelihood of selecting suppliers who are most likely to provide quality products and 
services, on time, and at reasonable cost/price. 

• Takes advantage of the experience and independent judgment of the source selection 
official. 

Potential Pitfalls 

• Using evaluation factors and subfactors that are not derived fi-om the market place 
and do not accurately reflect the Government’s requirements. This may result in 
award to an offeror that may not be the best value. 

• Using too many evaluation factors and subfactors. A large number of factors and 
subfactors dilutes consideration of those which are truly important. 

• Failure to make the appropriate investment in resources needed for a competent and 
defensible value analysis. 

• An inherently subjective process, and thus more difficult to evaluate and document. 

Major Steps in the Tradeoff Process 

The tradeoff process generally consists of the following steps: 

• Designing a strategy that best reflects the results of market research and the specific 
circumstances of the acquisition. 

• Establishing and documenting a source selection or technical evaluation plan. This 
plan includes the acquisition goals and objectives, identification and relative 
importance of evaluation factors and subfactors, the evaluation standards, and the 
selection process. 
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• Structuring the solicitation to effectively conununicate the Government’s 
requirements, mission objectives, the factors and subfactors, their relative 
importance, the information offerors must submit for evaluation against the stated 
factors and subfactors and the methodology for evaluating the proposals. 

• Evaluating the offers on the basis of the source selection plan and the evaluation 
factors and subfactors in the solicitation and having discussions as needed. 

• Comparing the strengths, weaknesses, risks, and cost/price or most probable costs of 
the proposals and deciding which combination, in accordance with the solicitation 
factors and subfactors, represents the best value. 

• Documenting the source selection decision including the tradeoffs and rationale 
used. 

• Awarding the contract, notifying offerors and debriefing them upon their request. 

• Documenting the lessons learned that may benefit future soiurce selections. 

The Lowest Price Technically Acceptable Process 

In some situations, simply comparing the cost or price of proposals meeting or exceeding the 
solicitation’s requirements for acceptability can be expected to result in the best value. In such 
cases, cost/price is the overriding consideration. While there may be a need for discussions there 
is no need to make tradeoffs. 

The lowest price technically acceptable process is similar to a sealed bid approach in that award 
is made to the acceptable offeror with the lowest evaluated cost or price. The major difference is 
that discussions can be held with offerors prior to source selection to ensure offerors understand 
the requirements and to determine acceptability. Tradeoffs are not permitted and no additional 
credit is given for exceeding acceptability. However, proposals are evaluated to determine 
whether they meet the acceptability levels established in the sohcitation for each noncost 
evaluation factor and subfactor. 

The lowest price technically acceptable process may be appropriate where the requirement is not 
complex and the technical and performance risks are minimal, such as acquisitions where 
service, supply, or equipment requirements are well defined but where discussions may be 
necessary. 

Major Steps in the Lowest Price Technically Acceptable Process 

The lowest price technically acceptable process generally consists of the following steps: 

• Designing a strategy that best reflects the results of market research and the specific 
circumstances of the acquisition. 

• Establishing and documenting a source selection or technical evaluation plan. This 
plan includes the acquisition goals and objectives, identification of acceptability 
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requirements for each noncost evaluation factor and subfactor, and procedures for 
evaluating proposals and making award. 

• Structuring the solicitation to effectively communicate the Government’s 
requirements, the factors and subfactors with associated acceptability standards, the 
information offerors must submit for evaluation of acceptability against the stated 
factors and subfactors, and the basis for award (i.e., the lowest priced proposal 
meeting or exceeding the standards.) 

• Evaluating and rating proposals on a pass/fail basis against the acceptability 
requirements in the solicitation. 

• Conducting discussions or other exchanges as needed. Comparing the cost or prices 
of acceptable proposals and awarding the contract to the offeror with the lowest 
evaluated price meeting the acceptability requirements. 

Past Performance and the Lowest Price Technically Acceptable Process 

• If you determine that past performance is a discriminator under this approach, then 
you must state in the solicitation the criteria that you will use to evaluate it on a 
pass/fail basis. For small businesses, an unacceptable rating in this area is a matter 
of responsibility. Therefore, in your acquisition planning, you should anticipate a 
possible need to obtain a Certificate of Competency from the Small Business 
Administration if a small business otherwise eligible for award has unacceptable past 
performance. 

• A Certificate of Competency determination is not required however, if you select a 
hybrid strategy that combines the lowest price technically acceptable and tradeoff 
processes. Under such a strategy, you could still evaluate technical proposals on a 
pass/fail basis while basing the final selection decision on a tradeoff between past 
performance and price. 

Conducting a Presolicitation Dialogue vyith Industry 
Foster a presolicitation dialogue with industry to: 

• Ensure a mutual understanding of the governments need and industry’s 
capabilities 

• Minimize inclusion of non-value added requirements, and 

• Promote a more effective source selection. 

• An effective dialogue with industry even before a solicitation is written or released 
can pay dividends during later phases of the process. The earlier and more effective 
you are in keeping up to date on the market and new technology, and ensuring the 
market knows what your requirements are, the better for both parties. The growing 
trend is to provide more information, not less, to potential offerors. With more 
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information, they can make informed decisions about whether to compete, they can 
offer better proposals, the evaluation and selection process will be quicker and 
smoother, and Aere is less chance of miscommunication and a protest. There are a 
variety of mechanisms to maintain contact with potential offerors including the 
following: 

Advanced Pl annin g Briefings for Industry to provide a forecast of future direction 
and requirements; 

Market research to stay abreast of irmovation, advances, and capabilities; 

Inf ormation centers to provide access information to documents relevant to the 
acquisition; 

Requests for Information and Draft Requests for Proposals to obtain information 
fi-om industry on such things as price and availability and comments on the proposed 
solicitation; 

Meetings and conferences, including one-on-one meetings with potential offerors 
and Presolicitation Conferences. 

In conducting a presolicitation dialogue with industry, always make sure that you: 

Release information to all potential offerors on a fair and equitable basis consistent 
with regulatory and legal restrictions. 

Establish clear ground rules for the conduct timing, and documentation of any one- 
on-one meetings to ensure potential offerors are given equal access to information 
needed to prepare proposals. 

Protect any proprietary information that you are given access to during this process. 

Request contracting and legal counsel advice if any questions arise about 
presolicitation exchanges. 


Drafting a Source Selection/Evaluation Plan 


A thoroughly contemplated plan for selecting a best value source is vital to any source selection 
process. In all source selections, the plan is tailored to reflect the complexity of the acquisition. 
In more complex source selections, this plan is called the Source Selection Plan and should be 
prepared for the source selection authority’s approval. In less complex acquisitions the plan is 
often referred to as the Technical Evaluation Plan. The plan is developed prior to or 
concurrently with preparation of the solicitation. It states your intentions for orgamzing and 
conducting the evaluation and analysis of proposals and the source selection. It contains 
acquisition sensitive information and is not released outside the contracting activity’s source 
selection organization. 
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Acquisitions using a tradeoff process are often subject to dynamic internal and external 
influences. Examples of such influences include: 


The differing missions or functions to be supported. Such situations influence how 
the agency specifies its requirements, which in turn influence offerors’ solutions. 

The rate at which technology and market factors are changing. Between the time the 
agency identifies a requirement and the offerors submit proposals, technology may 
have developed efficiency and productivity benefits unanticipated by the agency. 
Accordingly, you should structure the selection plan and the solicitation to consider 
these influences and assure that the proposal selected provides the best value to the 
government. 


Purpose of the Source Selection Plan 

The source selection plan serves several purposes, including — 

• Defining a specific approach for soliciting and evaluating proposals. 

• Describing the evaluation factors and subfactors, their relative importance, and the 
methodology used to evaluate proposals. 

• Providing essential guidance to the solicitation developers, especially for putting 
together the solicitation sections dealing with proposal preparation and evaluation. 

• Serving as a charter and guide for the source selection team on the roles of the 
members and the conduct of the entire source selection from proposal evaluation, 
through the cost/price/technical tradeoff, award decision, and debriefing. 

Guidelines for a Source Selection Plan 


Source Selection/Evaluation Plan 
Section I. Summary Description 

A. Item/Service to be Acquired 

B. Proposed Contract Type (Choose one) 

_FFP 

_FP with Incentives 

_Cost 

_Cost with Incentives 

_Hybrid (Explain) 

C. Source Selection Process (Choose one) 

Tradeoff 
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_Low Price Technically Acceptable 

_Hybrid (explain) 

D. Special Evaluation Techniques (explain, if applicable) 
e.g. oral presentations, multi-step techniques 

Section IL Source Selection Team 

A. Members. List the evaluation team members by name and functional area (including 
advisors, if applicable). 

B. Team Member Certifications: Verify/attach by reference certificates from each team 
member protecting unauthorized release of source selection or proprietary information. 

Section III. Evaluation Factors and Subfactors 

A. Proposal Evaluation Information. State all factors, subfactors and their relative order of 
importance exactly as they will appear in the solicitation. If you elect to include desirable 
objectives or features you would be willing to pay extra for, include an explanation of how they 
will be evaluated and whether or not credit will be given for exceeding such desirables. 

B. Proposal Submission Information. State the instructions to offerors for preparing and 
submitting proposals exactly as they will appear in the solicitation. 

Section IV. Description of Evaluation Procedures 

A. Rating System. Describe the evaluation technique (ratings and their definitions) that will be 
used to rate the proposals. 

B. Evaluation Standards. Describe the standards or target levels you will use to measure how 
well a proposal addresses each evaluation factor and subfactor. 

C. Procedures for Recording the Evaluation 

e.g. Worksheet(s) for recording evaluator ratings with supporting narrative 

D. Schedule. Identify the schedule for key source selection activities. 

Section V. Rationale for the Evaluation Scheme 

A. Factors and Subfactors. Explain how the selected factors and subfactors reflect the 
circumstances of your particular acquisition and the results of market research and other 
presolicitation exchanges with industry. 


B. Weights. Explain how the relative importance of the factors and subfactors reflect the 
acquisition’s primary objective. Explain why the most heavily weighted factors are the “drivers” 
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of the source selection. 


Blueprint; Typical Evaluation Plan 

Although there isn’t a specific format for the source selection plan, its size and detail should 
reflect tilie complexity of the acquisition. You should include, at a minim^lm, a discussion of the 
following: 

• A description of what you are buying. This description should be stated in fimctional 
terms to the maximum extent possible and use a minimum of technical language. 

• A description of the evaluation organization structure. It may be helpful to include -- 

An organization chart, showing the evaluation team’s structure, or a brief 
description of how the team is organized. 

The duties and responsibilities of each element of the source selection team. 

The evaluation team’s agenda and schedule. 

Information on the need for preparation and training of the evaluation team. 

Security procedures to be used by the evaluation team to protect classified, 
proprietary, or source selection information. 

CERTIFICATE FOR PERSONNEL PARTICPATING IN SOURCE SELECTION 
CONCERNING NONDISCLOSURE, CONFLICTS OF INTEREST, 

AND RULES OF CONDUCT 

Name:_ Organization:_ 

Title:_ Source Selection:_ 

1. I acknowledge that I have been selected to participate in the source selection identified above. 

I certify that I will not knowingly disclose any contractor bid or proposal or source selection 
information directly or indirectly to any person other than a person authorized by the head of the 
agency or the contracting officer to receive such information. I imderstand that unauthorized 
disclosure of such information may subject me to substantial administrative, civil and criminal 
penalties, including fines, imprisonment, and loss of employment under the Procurement 
Integrity Law or other applicable laws and regulations. 

2. To the best of my knowledge, I certify that neither I nor my spouse nor my dependent 

children, nor members of my household, nor persoimel with whom I am seeking employment 
have any direct or indirect foancial interest in any of the firms submitting proposals, or their 
proposed subcontractors or have any other beneficial interest in such firm except as fully_ 
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disclosed on an attachment to this certification. 

3. I certify that I will observe the following rules of conduct: 

a. I will not solicit or accept, directly or indirectly, any promise of future employment or 
business opportunity from, or engage, directly or indirectly, in any discussion of future 
employment or business opportunity with, any officer, employee, representative, agent, or 
consultant of a competing contractor. 

b. I will not ask for, demand, exact, solicit, seek, accept, receive, or agree to receive, directly or 
indirectly, any money, gratuity, or other filing of value firom any officer, employee, 
representative, agent, or consultant of any competing offeror for this acquisition. I will advise 
my famil y that the acceptance of any such gratuity may be imputed to me as a violation, and 
must therefore be avoided. 

c. I will not discuss evaluation of source selection matters with any unauthorized individuals 
(including Government personnel), even after contract award, ivithout specific prior approval 
firom proper authority. 

d. I understand that my obligations imder this certification are of a continuing nature. If at any 
time during the source selection process, I receive a contract fixim a competing contractor 
concerning employment or other business opportunity, the offer of a gift fi-om a competing 
contractor, or I encounter circumstances where my participation might result in a real, apparent, 
or potential conflict of interest, I will immediately seek the advice of an Ethics Counselor and 
report the circumstances to the Source Selection Authority. 

I understand that making a false, fictitious, or firaudulent certification may subject me to 
prosecution under Title 18, United States Code, Section 1001. 

Signature:_Date:_ 


Blueprint; Certifications 

• Plans for presolicitation activities such as issuing a draft solicitation and holding a 
presolicitation and/or preproposal conference or Advance Planning Briefing for 
Industry. 

• An acquisition strategy summary that includes an explanation of the contract type to 
be used (e.g., firm fixed price). 

• The proposed evaluation factors and subfactors, their relative importance, and 
associated evaluation standards. 

• A description of the evaluation process you are using (i.e., lowest price technically 
acceptable, tradeoff, or hybrid) and any innovative techniques such as multiple 
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phases or oral presentations, or tailoring. See Appendix E for details on oral 
presentations. Include a description of the rating system you are using. 

• A schedule of significant milestones that should cover, at a minimum, the period 
beginning with the designation of the source selection authority and continuing 
through Ae period from receipt of proposals through the signing of the contract, 
during which evaluation, negotiation, and selection take place. 

Selecting Evaluation Factors and Subfactors 

You must clearly state in the solicitation and source selection plan all the evaluation factors and 
subfactors that you will consider in making the source selection and their relative importance. 
These factors and subfactors inform offerors of all the significant considerations in selecting the 
best value source and the relative importance the Government attaches to each of these 
considerations. Offerors should understand the basis upon which their proposals will be 
evaluated and how they can best prepare their proposals. 


Evaluation factors help offerors 
understand the evaluation process 


Structure evaluation factors and subfactors and their relative order 
of importance to clearly reflect the Government’s need andfacilitate 
preparation of proposals that best satisfy that need. 

A multidisciplined team chooses the evaluation factors and subfactors based on user 
requirements, acquisition objectives, perceived risks, and thorough market research. Thorough 
research of the market helps the team identify the capabilities of different industry sectors and 
where those capabilities are most likely to differ among potential offerors. The team then selects 
only those factors that will help differentiate among offerors and surface the most advantageous 
proposal. 


Limit evaluation factors and subfactors to those areas that wUl reveal 
substantive differences or risk levels among competing proposals. 


Limit evaluation factors 
to true discriminators 


Cost Factors 

The Competition in Contracting Act (CICA), as implemented in the FAR, requires that price or 
cost to the Government be included as an evaluation factor in every source selection. TTiis is 
because affordability must always be a consideration when spending taxpayer dollars. 
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Always include cost or price 
as an evaluation factor 


The relative imp ortance between cost or price and the noncost factors must also be reflected in 
both the solicitation and the weights or priority statements in the source selection plan. 

However, cost/price is not numerically scored in the evaluation of proposals, because of possible 
distortions that can result when arbitrary methods are used to convert cost/price into scores. 

Cost-related factors and considerations will vary depending on the type of contract. Regardless 
of contract type, reasonableness must always be a consideration, as the FAR requires that 
contracts be awarded only at prices or costs that are fair and reasonable. 

Cost reaUsm plays an important role in many source selections. A cost realism analysis is an 
independent review of each offeror’s cost proposal to determine if specific estimated 

proposed cost elements are reahstic for the work to be performed; reflect a clear understanding 
of the requirements; and are consistent with the unique methods of performance and materials in 
the offeror’s technical proposal. 

Cost realism must be considered when a cost reimbursement contract is anticipated. Under a 
cost type contract, the proposed cost estimates may not be valid indicators of final actual costs 
that the Government will be obligated to pay. For this type of contract, a cost realism analysis is 
performed and used to determine the probable cost of performance for each offeror. Selection 
decisions should be based on these probable cost estimates. Significant differences between 
proposed and most probable costs may signal increased performance risks. 

Cost realism may also be considered for fixed price incentive contracts or, in exceptional cases, 
for other fixed price type contracts especially when there are concerns that offerors may try to 
“buy in” or where other complexities of the acquisition could result in misunderstanding the 
requirements. In such cases, a cost realism analysis may be useful for determining if there is a 
significant risk of future performance because of uiu-ealistically high or low prices. However, 
proposed fixed prices are not adjusted for cost realism during the evaluation. 

The solicitation must clearly state what costs will be evaluated. These costs may include costs 
for the basic effort only, basic plus all options, or costs incurred as a result of acquiring or 
owning an item (e.g., transportation, life cycle costs). The solicitation should also clearly 
indicate to offerors how the cost factor will be assessed for that acquisition. 

Past Performance 

The caliber of a contractor’s performance on previous contracts shall be included as an 
evaluation factor in competitively negotiated acquisitions unless the contracting officer 
documents why it would not be appropriate for the specific circumstances of the acquisition. A 
thorough evaluation of past performance, to include information that is outside of the offerors’ 
proposals, serves to ensure that awards are made to good performers rather than to just good 
proposal writers. See Appendix D for details on evaluating past performance. 
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Technical Factors 


Technical evaluation factors address the proposal’s technical and performance efficiency. These 
factors may include such considerations as technical approach and capabilities, management 
approach and capabilities, experience and personnel qualifications relative to satisfying critical 
aspects of the government’s requirements. Technical factors must be developed specifically for 
each acquisition, taking into consideration the particular objectives and requirements of the 
acquisition. These factors should be those discriminators that are determined after thorough 
market research as most likely to reveal substantive differences in technical approaches or risk 
levels among competing proposals. 

The source selection team has broad discretion in determining the technical evaluation factors 
and subfactors, their relative importance, and the way in which they will be applied. 

However, too many factors and subfactors can lead to a leveling of ratings, in which the final 
result may be a number of closely rated proposals with little discrimination among competitors. 

It is not the number of noncost factors that is critical, but having the right factors. 

Basic requirements for noncost evaluation factors are: 

• A reasonable expectation of variance among proposals in that area. 

• A variance that you can measure either quantitatively or qualitatively. 

• The factor must be a true discriminator. 

An evaluation factor should be chosen only if your requirements warrant a comparative 
evaluation of that area. The simplest way to assess a potential evaluation factor is to ask: “Will 
superiority in this factor provide value to the Govenunent and is the Government willing to pay 
more for that superiority?” 


Best Practices 

Selecting the right evaluation factors is one of the most important decisions you will make in 
designing your evaluation process. We are often faced with the triple problems of less time, less 
funds, and fewer available personnel to devote to source selections. If you don’t concentrate on 
what’s imp ortant in selecting the best value offeror you could end up with the evaluation team 
wasting a lot of time and effort looking at issues that don’t differentiate between offerors. This 
can also result in a weak evaluation that doesn’t give the source selection authority the 
information needed to make a good selection. 

There are certain factors that you must consider in any competitive source selection. Price/cost 
is an automatic factor that you always have to consider. You also have to consider past 
performance in your evaluation process imless the contracting officer documents why it is not 
appropriate for ^e specific circumstances of the acquisition. In addition, you may have to add 
factors that are required by regulation for specific acquisitions, such as any applicable 
preferences for small entities. From here, you add other factors and subfactors that are important 
to deciding which is the most advantageous proposal. Remember, not everything that the offeror 
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has to do under the contract is really a discriminator that will help you decide which proposal 
will result in the best value. Consider what you are buying and what will really discriminate. 

How to select the additional factors/subfactors? Consider the following methodology: 

• Research the market for what you are buying and your probable xmiverse of offerors. 

• Form an Integrated Product Team (IPT) and brainstorm critical factors and 
subfactors. 

• Select only those factors and subfactors likely to surface the most advantageous 
proposals. 

• Define the key discriminators and prioritize the list. 

• Get source selection authority approval of the list of factors/subfactors. 

• Clearly and concisely tell offerors in the solicitation what the factors/subfactors are 
and their relative importance. 

• Listen carefully to industry feedback firom presolicitation exchanges to see if your 
choices are right. If necessary, change the factors/subfactors before solicitation. 

Weighting the Factors and Subfactors 

After determining the evaluation factors and subfactors, their relative importance to each 

other must be established. The relative importance of factors and subfactors must be consistent 
with the stated solicitation requirements. If their relative importance does not accurately reflect 
the Government’s requirements and objectives, the source selection authority may later award to 
an offeror whose proposal may not be the best value. As a general rule, the higher the technical 
or performance risk, the greater the emphasis on noncost factors. The relative importance 
between all noncost factors combined and cost or price must also be described using the terms, 
“significantly more important,” “approximately equal,” or “significantly less important.” This 
relative ranking must be reflected in both the solicitation and the weights or priority statements 
in the source selection plan. 

The relative importance of evaluation factors and subfactors is usually established by priority 
statements, numerical weighting, or a combination of these. 

• Priority or tradeoff statements, numerical weighting, or a combination of these 
usually establishes the relative importance of evaluation factors and subfactors. 

• Priority or tradeoff statements would relate one factor to others. For example, in a 
priority statement, the cost/price factor may be said to be slightly more important 
than a noncost factor called “performance risk” but slightly less important dian a 
noncost factor called “technical merit.” 


132 



• Numerical weighting would involve assigning relative importance to the factors and 
subfactors using points or percentages. Although numerical weights may be used in 
making the tradeoff analysis and decision, the weights themselves may, but need not 
be disclosed in the solicitation. If you don’t disclose the numerical weights 
themselves in the solicitation, they must be described in terms of priority or tradeoff 
statements. 

Cost/price as an evaluation factor is never scored or rated as part of the evaluation. But, just like 
all the other factors and subfactors, cost/price has to be weighted to indicate its importance 
relative to the other evaluation factors and subfactors and the overall evaluation. The weight 
given to cost/price reflects its relative importance in selecting the best proposal for award. The 
circumstances of your particular acquisition will indicate how important cost/price is in 
satisfying yom requirement. 


Developing Evaluation Standards 

Evaluators must be able to determine the relative merit of each proposal with respect to the 
evaluation factors. Evaluation standards provide guides to help evaluators measure how well a 
proposal addresses each factor and subfactor identified in the solicitation. Standards permit the 
evaluation of proposals against a uniform objective baseline rather than against each other. The 
use of evaluation standards minimiz es bias that can result jfrom an initial direct comparison of 
proposals. Standards also promote consistency in the evaluation by ensuring that the evaluators 
evaluate each proposal against the same baseline. In developing standards for each evaluation 
factor and subfactor, you should consider the following: 

• As you develop yom evaluation factors, concurrently draft a standard for each factor 
and subfactor. 

• Define the standard by a narrative description that specifies a target performance 
level that the proposal must achieve in order to meet the standard for the factor or 
subfactor consistent with the requirements of the solicitation. 

• Describe guidelines for higher or lower ratings compared to the standard “target.” 

• Overly general standards should be avoided because they make consensus among 
evaluators more difficult to obtain and may obscure the differences between 
proposals. A standard should be worded so that mere inclusion of a topic in an 
offeror’s proposal will not result in a determination that the proposal meets the 
standard. (An example is shown at Appendix C.) 

• While it is sometimes easier to develop quantitative standards because of their 
definitive nature, qualitative standards are commonly used in source selections. 
Standards, as part of the source selection methodology, should be included in the 
source selection plan. 

Establishing a Rating Method 
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Typical Ratings and Descriptors 

Each rating must have a definition. 


TECHNICAL MERIT ratings reflect the government’s confidence in each offeror’s ability, as 
demonstrated in its proposal, to perform the requirements stated in the RFP. Choose one method 
(e.g., numerical, adjectival, or color) to evaluate technical merit. 

NUMERICAL 

ADJECTIVAL COLOR 

DEFINITION 

90-100 

Excellent Blue 

Proposal demonstrates excellent understanding 
of requirements and approach that significantly 
exceeds performance or capability standards. 

Has exceptional strengths that will 
significantly benefit the Government. 

80-89 

Good Green 

Proposal demonstrates good understanding of 
requirements and approach that exceeds 
performance or capability standards. Has one 
or more strengths that will benefit the 
Government. 

70-79 

Satisfactory Yellow 

Proposal demonstrates accqptable 
imderstanding of requirements and approach 
that meets performance or capability standards. 
Acceptable solution. Few or no strengths. 

60-69 

Marginal Amber 

Proposal demonstrates shallow understanding 
of requirements and approach that only 
marginally meets performance or capability 
standards necessary for minimal but acceptable 
contract performance. 

<60 

Unsatisfactory Red 

Fails to meet performance or capability 
standards. Requirements can only be met with 
major changes to the proposal. 

PROPOSAL RISK ratings assess the risks and weaknesses associated with each offeror’s 
proposed approach to performing the requirements stated in the RFP. It is an overall assessment 
derived firom the technical evaluation and is driven by each of the subfactors within the technical 
factor. 

ADJECTIVE 

DESCRIPTION 


Low Risk 

Any proposal weaknesses have little potential to cause disruption of schedule, 
increase in cost, or degradation of performance. Normal contractor effort and 
normal Govermnent monitoring will probably minimize any difficulties. 
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Moderate Risk 

Approach has weaknesses that can potentially cause some disruption of 
schedule, increase in cost, or degradation of performance. However, special 
contractor emphasis and close Government monitoring will probably 
minimize difficulties. 

High Risk 

Approach has weaknesses that have the potential to cause serious disruption 
of schedule, increase in cost, or degradation of performance even with special 
contractor emphasis and close Government monitoring. 

PERFORMANCE RISK (Past Performance) ratings assess the risks associated with each 
offeror’s likelihood of success in performing the requirements stated in the RPP based on that 
offeror’s demonstrated performance on recent, relevant contracts. 

ADJECTIVE 

DESCRIPTION 

Very Low Risk 

Offeror’s past performance record provides essentially no doubt that the 
offeror will successfully perform the required effort. 

Low Risk 

Offeror’s past performance record provides little doubt that the offeror will 
successfully perform the required effort. 

Moderate Risk 

Offeror’s past performance record provides some doubt that the offeror will 
successfully perform the required effort. 

High Risk 

Offeror’s past performance record provides substantial doubt that the offeror 
will successfully perform the required effort. 

Very High Risk 

Offeror’s past performance record provides extreme doubt that the offeror 
will successfully perform the required effort. 

Unknown Risk 

The offeror has no relevant performance record. A thorough search was 
unable to identify any past performance information. 

COST - NOT “RATED.” Reflects the evaluated cost. RFP must describe method by which cost 
will be evaluated (e.g., how probable cost or life cycle cost will be evaluated.) 


Blueprint; Typical Ratings and Descriptors 

A rating system uses a scale of words, colors, numbers or other indicators to denote the degree to 
which proposals meet the standards for the noncost evaluation factors. Thus, a rating system 
helps evaluators assess a proposal’s merit with respect to the evaluation factors and subfactors in 
the solicitation. Some commonly used rating systems are adjectival, color coding, and 
numerical. 

Rating systems which use adjectives or colors are usually the most successful because they allow 
maximum flexibility in making the tradeoffs among the evaluation factors. A narrative 
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definition must accompany each rating in the system so that evaluators have a common 
understanding of how to apply the rating. For example, a rating of excellent (or blue or 90-100) 
could be defined as meaning an outstanding approach to specified performance with a high 
probability of satisfying the requirement. What is key in using a rating system in proposal 
evaluations, is not the method or combination of methods used, but rather the consistency with 
which the selected method is applied to all competing proposals and the adequacy of the 
narrative used to support the rating. 


Adjectival 

Adjectives (such as excellent, good, satisfactory, marginal, and imsatisfactoiy) are used to 
indicate the degree to which the offeror’s proposal has met the standard for each factor 
evaluated. Adjectival systems may be employed independently or in connection with other 
rating systems. 


Color Coding 

This system uses colors to indicate the degree to which the offeror’s proposal has met the 
standard for each factor evaluated. For instance, the colors blue, green, yellow, amber, and red 
may indicate excellent, good, satisfactory, marginal, or unsatisfactory degrees of merit, 
respectively. 


Numerical 

This system assigns point scores (such as 0-10 or 0-100) to rate proposals. This rating system 
generally allows for more rating levels and thus may appear to give more precise distinctions of 
merit. However, numerical systems can have drawbacks as their apparent precision may obscure 
the strengths, weaknesses, and risks that support the numbers. Therefore, some organizations do 
not permit the use of numerical rating systems. 

Narrative 

Narrative is used in conjimction with a rating system to indicate a proposal’s strengths, 
weaknesses, and risks. Adjectival, color, and numerical ratings must be supported with narrative 
statements. Narrative statements can describe the proposals’ relative strengAs, weaknesses, and 
risks to the source selection authority in a way that adjectives, colors, and numbers alone cannot. 
A narrative is required when evaluation standards are being applied, when a comparison of 
proposals is being made, and when a cost/techmcal tradeoff is conducted. The narrative provides 
a reasonable and rational basis for the selection decision. 
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The Solicitation 


Ensure consistency among the objectives of the acquisition, 
the contracting strategy, the evaluation plan, the solicitation, 
the evaluation and selection. 

All the parts of the solicitation work together to communicate government requirements to 
potential offerors. The solicitation provides all the information the offeror needs to understand 
what you are buying, how you are buying it, and how you will select who to buy it from. This 
information includes: the work requirements; the terms and conditions; evaluation factors and 
significant subfactors; the relative importance of the factors and subfactors; instructions to 
offerors, including whether award might be made without discussions; and other exhibits and 
attachments. When read as a whole, the solicitation should convey to the offerors a clear 
understandmg of what you are buying and the areas where technical and cost tradeoffs can be 
made in their proposals to best satisfy the Government requirements. 

Industry frequently complains that solicitations have major conflicts. Particularly troublesome 
are conflicts among the descriptions of what we’re buying, instmctions on how to prepare a 
proposal, and guidance on important factors/subfactors and the ground rules for the evaluation. 
An inconsistent solicitation may result when different groups of people develop the different 
sections without proper coordination. Such a solicitation can defeat our objectives, cause 
unnecessary delays, or lead to litigation. 

Coordination within a multidisciplined acquisition team, whose members are stakeholders in the 
acquisition and have a commitment to work together, is the best way to ensure consistency. You 
may also find it beneficial to develop a matrix that correlates the solicitation sections and content 
to ensure solicitation consistency. You may want to provide industry with a copy of the matrix 
as a reference tool to aid in proposal preparation. This approach promotes understanding of the 
linkage within the solicitation and explains how all parts of the proposal will be used in the 
evaluation process. 

Appendix C illustrates how the key solicitation documents and evaluation standards track to one 
another and shows the recommended sequencing for dociraient preparation. 

Another way to promote imderstanding of the solicitation is to foster a presolicitation dialogue 
with industry. 

This can be accomplished through use of various communication forums such as Commerce 
Business Daily notices. Advance Planning Briefings for Industry, draft solicitations, and/or 
presolicitation/preproposal conferences. 

Performance Requirements — (Specifications, Work Statement, or Equivalent) 

The way you present the Government’s requirements in the solicitation can have a significant 
impact upon a source selection using the tradeoff approach. For example, use of a work 
breakdown structure (WBS) in the work statement for the most complex cost type contracts can 
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help ensure offerors’ pricing breakdowns are consistent and comparable. Some additional areas 
to consider when preparing the work requirements for the solicitation include: 


Functional or Performance Requirements 

Use functional or performance requirements to the maximum extent possible. In some cases, it 
may be more difficult to develop evaluation standards and conduct the evaluation process itself; 
however, there are benefits to using functional or performance requirements. These benefits 
include: 

• Increased competition. 

• Access to the best commercial technology. 

• Better technical solutions for better prices as a result of offeror iimovation. 

• Functional or performance requirements can usually be developed faster than design 
requirements. 

• Fewer situations may exist for protests. 


Using performance requirements 
can lead to offeror innovation 


Design Requirements 

You should limit the number of design requirements to those essential to meet mission needs. 
Design requirements may: 

• Limit competition. 

• Limit situations where potential offerors can propose innovative solutions. 

• Slow the specification development process. 

• Provide more situations for an offeror to protest (e.g., because of the behef that the 
winning proposal did not meet all the minimum requirements or that the 
requirements were unnecessarily restrictive of competition). 

Proposal Submission Information 

The instructions for preparing and submitting proposals are critical to an acquisition using the 
tradeoff approach. There has to be a linkage between solicitation requirements, each evaluation 
factor and subfactor and the proposal preparation instructions. 
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Each evaluation factor and subfactor 
must correlate directly with the 
proposal preparation instructions 


If you cannot cross-walk the solicitation requirements, factors/subfactors and the proposal 
instructions, you have a conflict that you need to correct. 

Request only the information needed to evaluate proposals 
against the evaluation factors and subfactors. Never ask 
for information you do not intend to evaluate. 

The information requested from offerors must correlate with the evaluation factors and 
subfactors. However, instructions that require voluminous information can cause potential 
offerors to forego responding to the solicitation in favor of a less costly business opportunity. 
Furthermore, excessive size of proposals may increase the CJovemment’s costs to perform the 
evaluation and length of the evaluation period. In order to simplify the preparation of proposals 
and to make the evaluation easier, you may wish to consider imposing a realistic limit on the 
number of pages and foldouts to be submitted. 

The instructions on the preparation and submission of proposals must: 

• Be clearly and precisely stated. 

• Be keyed to the evaluation factors and subfactors. 

• Describe the type, scope, content, and format of the information to be submitted. 

• Describe the order in which proposal responses and materials are to appear. 

• Be limited to the information needed to do the evaluation. 

• Properly written proposal preparation instructions simplify the evaluators’ job. That 
is, evaluators do not have to learn a new format for each proposal; they can evaluate 
the same requirements in each proposal in the same way. With a sufficient degree of 
structure in the proposal preparation requirements, you may be able to accept 
proposals in electronic form and use some automation in the evaluation process. 

Proposal Evaluation Information 

Clearly state in the solicitation the basis upon which the 
Government will make the source selection decision. 

The information from the Source Selection/Evaluation Plan that you provide in the solicitation 
on evaluation factors and subfactors and their relative importance forms the basis for evaluating 
offerors’ proposals and making the cost/technical tradeoff. The solicitation is the official vehicle 
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for you to communicate to offerors which factors and subfactors or groimd rules the Government 
will use to select the most advantageous proposal for award. 

Consider the following points in designing the solicitation: 

• Provide the evaluation factors and subfactors verbatim jfrom the source selection 
plan. 

• Provide the actual numerical weights at the factor level. 

• Provide an estimate of what you’ve identified as an affordable target price range for 
the acquisition, based on your market research or other reviews. 

This information can help offerors to better focus on those aspects of the mission objectives 
where additional value can be important and to better respond to the Government’s needs by 
giving emphasis to those things most important to the Government. To reap the benefits of 
better proposals you need to include and adequately describe all the factors and subfactors (as 
reflected in the source selection plan) that will be considered in making the selection. 

The solicitation must also inform offerors of any minimum requirements that apply to particular 
evaluation factors and subfactors that have to be met. You need to distinguish between 
minimum acceptable requirements and desirable objectives or features that you would be willing 
to pay extra for. If you elect to include desirable objectives or features in addition to minimum 
requirements, the solicitation must clearly explain how you will evaluate them and whether or 
not credit will be given in the evaluation for exceeding such desirables. 
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Evaluation Considerations 


Overview 


The source selection evaluation process includes examining each proposal in detail against the 
evaluation factors and subfactors and the requirements set forth in the solicitation, and assigning 
a rating, with a supporting narrative. The proposal evaluation process assesses the proposal and 
the offeror’s ability to perform. At this stage, it does not analyze proposals against each other 
and it must be conducted in a fair, comprehensive, and impartial maimer. 


Evaluations must be fair, 
thorough, and impartial 


The evaluation process can be complicated no matter how much planning and tailoring you do. 
You might find through your market research that you are going to receive many proposals -- 
good for competition but a situation that could drag out evaluation. You might explore the world 
of automated source selection tools that can increase your efficiency. There are commercial 
packages available and some activities have developed packages in house that you could use. 
Evaluators still have to evaluate each proposal, but these tools might ease the admimstrative 
burden that comes with a great number of proposals. 

Reasons for the Evaluation Process 


The principal purposes of the process are to: 

• Determine which proposals are acceptable and/or within the competitive range. 

• Provide a soimd basis for the source selection authority to make an informed and 
reasoned selection by: 

Presenting a clear picture of the issues considered during evaluation by 
identifying areas of uncertainty as well as those which provide substantial 
assurance of a successful outcome. 

Listing the strengths, weaknesses, and risks of the proposed approaches. 
Evaluation Process Tasks 

Evaluation tasks will vary in number and content with each source selection. However, several 
especially important tasks are discussed below. 

Familiarization 
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Prior to receipt of proposals, each evaluator should become familiar with the solicitation’s 
requirements, the source selection plan, and the rating system. You should, especially for those 
evaluators with no prior source selection evaluation experience, conduct training that includes an 
overview of the solicitation and of the work expected tihroughout the source selection process. 
The training should include how to properly document each proposal’s strengths, weaknesses, 
and risks. 


Cost Evaluations 

Cost or price must be an evaluation factor in all acquisitions. The cost evaluation will vary 
depending on the specific circumstances of each acquisition. 

For fixed price contracts, the evaluation normally should be as simple as a comparison of the 
offered prices to ensure the contract price is fair and reasonable. Other techniques of price 
analysis may also be used. Do not perform a cost analysis unless there is no other way to 
determine if the price of the otherwise successful offeror is reasonable. 

For cost-reimbmsement contracts, you must analyze costs for both realism and reasonableness. 
The cost realism analysis enables you to determine the probable cost of performance for each 
offeror. This precludes an award decision based on overly optimistic offeror’s cost estimates 
where risks of an overrun may be significant. 

A cost realism analysis requires an independent review of specific elements of each offeror’s 
proposed cost estimate to determine whether the estimated proposed cost elements for contract 
resources (e.g., labor and material) are realistic, show understanding of the work, and are 
consistent with the demands of the work which will actually be required, given each offeror’s 
unique methods of performance and materials described in their technical proposal. 

The probable cost should reflect the Government’s best estimate of the cost of any contract, 
which is most likely to result fi’om the offeror’s proposal. This estimate is determined by 
adjusting each offeror’s proposed cost, and fee when appropriate, upwards or downwards to 
reflect any additions or reductions in personnel, equipment, or materials resulting jfrom the cost 
realism assessment. 

For the cost realism evaluation of an offeror’s proposal, you have to decide what information you 
need. The amount and type of information will vary depending on the circumstances of your 
acquisition. You may have to get more after you start evaluating the proposals. However, like 
other proposal requirements, you should only request the minimum amormt of information that is 
necessary. Also, remember that any information you use only for the cost realism analysis is not 
considered cost or pricing data. 

To the extent that differences between proposed costs and probable costs reflect significant risks 
of future performance or lack of understanding, that risk or lack of understanding should be 
reflected in the noncost evaluation. In such cases, you should also seriously consider whether or 
not the proposed cost and fee or price can be determined fair and reasonable to both parties. 

The probable cost estimates developed for each offeror are used to evaluate and compare 
proposals and ultimately to select the proposal expected to result in the best value. 
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Past Performance Evaluations 

Unless you are using a lowest price technically acceptable approach, the past performance 
evaluation involves a comparative assessment of performance risk associated with each proposal. 
It describes the degree of confidence the government has in the offeror’s ability to perform based 
on that offeror’s demonstrated record of past and present work similar to the work to be 
performed. If properly conducted, the past performance evaluation and the preaward survey will 
complement each other and provide a more complete picture of an offeror than either one could 
by itself 

Appendix D contains procedures for evaluating past performance in source selections, including 
those acquisitions where selection is based solely on cost/price and past performance. 

Technical Evaluations 

PROPOSAL EVALUATION WORKSHEET 


RFPNo: 

EVALUATOR’S NAME: I OFFEROR: 



FACTOR: | VOLUME/PARAGRAPH: 



SUBFACTOR: I PAGE NUMBER: 



Merit Ratings: (Refer to your rating definitions. 



Evaluator’s Rating: (Merit/Risk) 


Initial Rating: (e.g., G/M) Evaluator Initials/ Date: Team Leader Initials/Date: 

Discussions: Evaluator Initials/ Date: Team Leader Initials/Date: 

Final Rating: Evaluator Initials/ Date: Team Leader Initials/Date: 

RATIONALE: Include supporting rationale for the ratings. Using the evaluation standards and 
rating definitions, state the evaluation results in terms of strengths; weaknesses/significant_ 


143 

















weaknesses/deficiencies; and risks. Include questions that address ambigmties or other concerns 
(e.g., perceived deficiencies, weaknesses, omissions, mistakes). Identify all comments and 
questions below with the rating: (e.g.. Initial Rating (IR), Result of Discussions (RD) or Final 
Rating (FR). Use continuation sheets or a database as needed and a separate sheet for every 
factor and subfactor. 


STRENGTHS: 


WEAKNESSES/SIGNIFICANT WEAKNESSES/DEFICIENCIES: 

(Precede comment with an [S] if it identifies a significant weakness or a [D] if a deficiency.) 


RISKS: 


QUESTIONS: 


Blueprint: Typical Evaluation Worksheet 

Evaluators must examine each proposal individually in detail to measure it against the evaluation 
factors and subfactors in the solicitation. Evaluators ask questions such as, “How much?” or 
“How well?” assign a rating and document the basis for the rating. This is the core of the 
evaluation process. 

Normally, technical evaluations should be conducted independent of the cost/price evaluations so 
that technical findings and conclusions will not be influenced by knowledge of the offered costs. 
However, in some instances, it may be appropriate to give the entire evaluation team access to 
price/cost information to ensme the best possible overall evaluation and enhance the evaluation 
of cost realism. Such a review can help verify perceived technical strengths, weaknesses or risks 
and/or ensure consistency between the cost/price and technical segments of the proposals. 

All evaluators must have the required functional expertise and training to evaluate the particular 
area of the proposal to which they are assigned. They should also be thoroughly familiar with 
the solicitation and the source selection plan. 

Identifying Proposal Ambiguities and Inadequate Substantiation 
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Evaluators should first document problems in evaluating a proposal because its language is 
ambiguous, its meaning is unclear, or it has failed to respond to the solicitation instractions. 
Evaluators should also identify, in writing, instances in which an offeror has not provided 
enough information to evaluate the feasibility and merit of its proposed approach. The 
Contracting Officer can then seek amplification and additional information to address such 
issues. 


Identifying Strengths, Deficiencies, Significant Weaknesses, and Risks 

Evaluators must identify and document the strengths, deficiencies, significant weaknesses, and 
the accompan5dng risks of the competing proposals. Proposals that materially fail to meet a 
Government requirement or that contain a combination of significant weaknesses that increase 
the risk of imsuccessful performance are considered to be deficient. 

Narrative statements must be used to establish a written record. Nmnerical scores and other 
rating techniques are not conclusive data to make the sovurce selection decision. Only 
evaluations and ratings substantiated by specific strengths, weaknesses, and risks can be credible 
and justifiable. General terms such as “weak,” “poor,” or “excellent” must be supported with 
specific reasons as to why the proposal is “weak,” “poor,” or “excellent” in relation to the 
standard for the specific factor and subfactor being evaluated. 

The strengths, weaknesses, and risks of each proposal form a large part of the basis for the 
somce selection decision. 


Documenting proposal strengths, 
weaknesses, and risks is critical 


Generally, the fact that a proposal is deficient as submitted does not necessarily mean that it is 
excluded fi'om fiuther consideration. The identification of these vital items provides: 

• An element for the contracting officer to consider in determining the competitive 
range. 

• The framework for any necessary discussions between the Govermnent and the 
offeror. 

• Specific information on the relative strengths and weaknesses of competing 
proposals. This is critical to the successful completion of an acquisition using the 
tradeoff approach because it is an essential element of the evaluation report provided 
to the source selection authority. 

• The basis for tradeoff analysis ultimately performed by the source selection authority 
to determine if differences in merit between proposals justify any cost/price 
differential. 

• The fi’amework for offeror debriefings. 
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A separate evaluation finding, regardless of the offered cost or price, can be a determination that 
a proposal is technically imacceptable. This finding is based on failure to meet requirements, or 
even the basic intent of the acquisition, and that a complete revision of the proposal would be 
required. In this case, you would be put in the position of leading the offeror to a solution or 
approach, which is unfair to the other offerors. 

Consensus 

The final rating of each proposal should be assigned by consensus of the evaluators. Simple 
averaging of individual evaluation results does not constitute consensus. Consensus reqmres a 
meeting of the minds on classifications, deficiencies, strengths, weaknesses, and risks. In 
exceptional cases where the evaluators are unable to reach agreement without unreasonably 
delaying the acquisition process, the evaluation report may include the majority conclusion and 
the dissenting view(s), each with a supporting rationale. 

Exchanging Information with Offerors 

Dialogue with offerors after receipt of proposals allows us to get information we need to better 
understand proposals and make best value decisions. While all such dialogue must be conducted 
in a fair and impartial manner, its nature and extent will vary depending upon when it occurs 
after receipt of proposals. 

Who is in Charge? 

The contracting officer remains the focal point for all information exchanges with prospective 
contractors fi-om release of a solicitation through contract award. Once proposals are received, 
the contracting officer also controls all exchanges with offerors. 

Establishing the Ground Rules 

Before exchanging any information with offerors, the contracting officer should ensure that team 
members who may participate in such exchanges receive instmctions not to: 

• Favor one offeror over another (i.e., provide the offeror with suggested ways to 
correct its proposal relative to other offerors); 

• Reveal an offeror’s solution, technology, or intellectual property to another offeror; 

• Reveal an offeror’s price without that offeror’s permission; 

• Reveal the name of individuals providing past performance information; or 

• Knowingly furnish source selection information. 

Award Without Discussions 

Before issuing the solicitation, you must decide whether or not you intend to award without 
discussions and communicate your intent in the solicitation. In making this decision, consider 
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whether or not you are likely to obtain best value without discussions. An award without 
discussions is most likely to result in best value when requirements are clear, commodities are 
known or stable, and the marketplace is extremely competitive. 

If your solicitation advised offerors of intent to award without discussions, you may still hold 
discussions, if appropriate, provided you document the file as to why discussions are necessary. 

If, after proposal evaluation, it is clear that the cost of conducting discussions would more than 
offset the potentially lower prices or increased functionality resulting from discussions, then it 
may be appropriate to award on initial proposals. 

Requesting Clarifications When Awarding Without Discussions 

The most limited exchanges are clarifications that occur if award will be made without 
discussions. Under these circumstances, we may give offerors the opportunity to clarify certain 
aspects of their proposals such as questions about the relevancy of their past performance or 
adverse past performance information on which an offeror hasn’t yet had an opportunity to 
comment. These exchanges may be used to resolve minor irregularities, informalities, or clerical 
errors. Such clarifications provide minor explanations but do not revise or modify the proposal, 
except to the extent that correction of apparent clerical mistakes results in a modification. 

Holding Communications 

Before makin g a competitive range decision, you may need to hold communications with some 
offerors to determine whether or not to include a proposal in the competitive range. This is like 
fact-finding. The objective of these precompetitive range exchanges is to help evaluators 
understand and evaluate the proposal. 


Communications may be held 
to help evaluators understand 
gray areas in the proposal 


Communications must be held with any offeror who will be excluded from the competitive range 
because of their adverse past performance information. Otherwise, you may hold 
communications only with those offerors who are neither clearly in nor clearly out of the 
competitive range. If you know that you will include an offeror in the competitive range, then 
wait until you open discussions to address your concerns. 

Offerors should ensure that initial proposals are as clear and complete as possible. When holding 
communications, ask only those questions necessary to understand the proposal and make the 
competitive range determination. You may use communications to solicit information that will 
clear up gray areas, such as perceived deficiencies, omissions, and errors, or questions about an 
offeror’s capability or preaward survey. During communications, you must give offerors an 
opportunity to address any adverse past performance information to which the offeror has not 
previously had an opportunity to comment. This ensures that offerors are not excluded from the 
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competitive range on the basis of incorrect past performance information that they had not had a 
prior opportunity to address. 

Information obtained during communications, however, may not be used to revise a proposal, 
correct any deficiencies or material omissions, or change any technical or cost elements of a 
proposal, except for correction of mistakes. 


Communications do not 
permit proposal revisions 


Once you have enough information to decide how the proposal should be rated, (e.g., decided 
whether a potential deficiency is, indeed, a deficiency), then STOP. Never accept a revision 
before opening discussions. 


Establishing the Competitive Range 

The competitive range consists of all the most highly rated proposals, unless it is further reduced 
for efficiency. Establishing the competitive range results in greater efficiency by limiting the 
number of offerors with whom the Government must hold discussions to the finalists or leading 
contenders for contract award. However, failure to properly establish a competitive range can 
result in higher costs because of protests or eliminating potentially competitive offerors. When 
establishing the competitive range, consider the following points: 

• Determine the competitive range only after an initial evaluation of each proposal in 
accordance with all cost and noncost factors in the solicitation. 

• Limit the competitive range to all of the most highly rated proposals, considering the 
initial evaluation of both cost and noncost factors. Predetermined “cut-off’ ratings 
cannot be used to exclude a proposal fi’om the competitive range. 

• If there are very few highly rated proposals, you may want to include all of them in 
the competitive range. 

• If there are too many highly rated proposals to evaluate efficiently, you may limit the 
competitive range farther, provided you notified offerors of your intent to do so in 
the solicitation. 

• It may not always be necessary or even advisable to further narrow the competitive 
range for efficiency. You must determine what constitutes an efficient competitive 
range for each acquisition. When faced with the need to restrict the size of the 
competitive range, you should consider factors such as the expected dollar value of 
the award; the complexity of the acquisition and solutions proposed; or the extent of 
available resoiurces and other relevant matters consistent with the need to obtain the 
best value. 
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• When further reducing the competitive range for efficiency, select jfrom among the 
most highly rated proposals, the largest number that will still permit an efficient 
competition. 

• The contracting officer determines the competitive range. In the case of more 
complex source selections, the determination is made with the approval of the source 
selection authority. 

• Document the competitive range determination and the supporting rationale in the 
contract file. 

• Maintain an efficient competitive range that doesn’t waste resources for either side. 
The competitive range should be continually reassessed as discussions and 
eyaluations continue. The contracting officer should remove from the competitive 
range any proposal that, during or after discussions, is no longer considered to be a 
leading contender for award. This allows offerors who are not likely to be selected 
for award to shift their bid and proposal costs to competitions where they have a 
better chance for success. The objective is an efficient competitive range that 
doesn’t string offerors along wasting their time and money and your resources. 


Maintain an efficient competitive 
range that doesn’t waste resources 


• For proposals excluded from the competitive range, the contracting officer shall 
promptly notify unsuccessful offerors, in writing, of their exclusion. Upon request, 
you will also have to provide a debriefing that explains the basis for your decision. 
See Appendix F for more information on debriefings. 

Conducting Discussions 

The most detailed and extensive exchanges are negotiations that are held after establishment of 
the competitive range. These exchanges are known as discussions. Unless the solicitation 
informs offerors that award may be made without discussions, you must hold meaningful 
discussions with each offeror in the competitive range. 

The primary purpose of discussions is to maximize our ability to get the best value. 


Discussions maximize our 
ability to get the best value 


During discussions, our objective should be to reach complete agreement between and 
understanding by the Government and the offeror regarding all the basic requirements in the 
solicitation. In essence, obtaining a contract that demonstrates the greatest promise of meeting 
the solicitation’s requirements and no surprises after award is the goal of both the Government 
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and the offeror. While the content of discussions is a matter primarily within the discretion of 
the contracting officer, discussions must meet fundamental requirements to be meaningful and 
fair. 


Discussions must be meaningful and fair 


Ensure discussions are meaningful by identifying to the 
offeror all evaluated deficiencies, significant weaknesses, 
and other proposal aspects that could be altered or explained 
to enhance materially an offeror*s award potential 

Confine and tailor your discussions exclusively to each offeror’s proposal relative to the 
solicitation requirements and evaluation factors and subfactors. Identify those things in the 
proposal that could clearly limit an offeror’s award potential. Seeking fee advice of legal 
counsel during fee discussion process may help avoid protests. 

You can facilitate meaningful discussions by addressing fee following as a minimum: 

• Deficiencies -- A material failure to meet a requirement. It is a deficiency whenever 
fee offeror specifically says a requirement cannot or will not be met, offers an 
approach feat clearly doesn’t meet a requirement, or submits a proposal feat contains 
a combination of significant weaknesses. 

• Significant Weaknesses — Include noncost and cost weaknesses feat appreciably 
increase fee risk of unsuccessful contract performance. It is a weakness whenever 
fee proposal has a flaw important enough to cause a factor to be rated marginal or 
poor, or fee probability of meeting a requirement to be high risk or moderate to high 
risk. This includes even relatively minor weaknesses if their cumulative impact is 
significant. For example, if an approach affects several areas of the evaluation, but 
makes no individual factor rating marginal or poor, you should include it in 
discussions if fee cumulative impact is significant enough to impact fee overall 
rating. 

• Past Performance Information - Include any concern about an offeror’s past 
performance, including relevancy and any adverse past performance information on 
which fee offeror has not previously had an opportunity to comment. 

• Uncertainties or apparent mistakes - Include any suspected errors, any significant 
omissions, and any uncertainties necessary to understand what is being offered. 
However, perfect knowledge isn’t necessary. We tend to spend too much of our time 
and effort and feat of fee offeror, chasing information feat has no real bearing on fee 
evaluation. If we need it to draw a conclusion, then we should ask for it. 

Identify deficiencies and sigmficant weaknesses in terms of a clear declarative statement. 
Advising offerors of strengths in their proposals can also give offerors insight into areas to 
consider in making tradeoffs to correct deficiencies or weaknesses. 
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Obtaining Proposal Revisions 


Confirm all information obtained through discussions by requesting or allowing proposal 
revisions, as appropriate, firom all offerors in the competitive range still eligible for selection. 
Proposals are rarely alike, nor are the depth and range of discussions, therefore, tailor the number 
and content of revisions to each offeror’s proposal. Ask offerors to submit written changes to 
their proposals resulting fi'om discussions before requesting final proposal revisions, particularly 
if a number of significant issues need resolution. TWs allows further discussions, if necessary 
before the final cutoff date. 


Tailor the number and content of 
revisions to each offeror’s proposal 


After you have received responses to all issues raised to the offerors during discussions, you 
must reevaluate the proposals. Any factor impacted by the responses must be rated again in the 
same manner as in the initial evaluation. Ensure that all issues are resolved or understood by 
each offeror and the government prior to concluding discussions. 

At the conclusion of discussions, you must give all offerors remaining in the competitive range 
an opportunity to improve their proposal by submitting a final proposal revision within a 
common cutoff date and time. If, after receipt of final revised proposals it becomes necessary to 
subsequently clarify minor irregularities, you can, without any additional request for final 
proposal revisions firom all offerors. However, if you need to negotiate further, a second final 
revision opportunity must be extended to all offerors. 
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Selection Decision 

Ensure the selection decision: 

• Is based on a comparative analysis of the proposals; 

• Is consistent with stated evaluation factors and subfactors; and 

• When tradeoffs are permitted, consider whether or not perceived benefits are 
worth any price premium. 

• Make the decision on a rational basis and set it forth in an independent, 
stand-alone defensible document 

Consistent with the solicitation, after the team has completed the evaluation of the individual 
proposals, the source selection authority compares competing proposals to each other. 

When using the lowest price technically acceptable process, the source selection authority 
compares proposals on the basis of cost or price alone and selects the offeror with the lowest 
evaluated cost'price meeting the acceptability requirements for all factors and subfactors. 

When using the tradeoffprocess, the source selection authority compares proposals on the basis 
of cost/price, technical or other noncost ratings, and how its strengths, weaknesses, and risks will 
impact the specific objectives of the acquisition. The source selection authority may request the 
evaluators to conduct comparative analyses of proposals and make a recommendation 
concerning the source selection. The source selection authority will use all the information on 
the proposals and evaluation to make an independent judgement of the best value. 

Consistent with the solicitation, the possible outcomes of this comparison are: 

• The proposal with the superior noncost merit is the lowest cost/price proposal. In 
this case award should be made to the offeror submitting the proposal with the 
lowest evaluated price or cost. 

• The proposals may be determined to be essentially equal in terms of noncost factors. 
In this case also, award should be made to the offeror submitting the proposal with 
the lowest evaluated price or cost. 

• When the proposal with the lowest evaluated price or cost is other than the 
proposal(s) with higher noncost merit, the source selection authority must perform a 
cost/technical tradeoff analysis to decide whether the technical superiority of the 
other proposal(s) warrants payment of the additional price or cost. 

Makin g the Cosl/Technical Tradeoff Analysis 

Ratings are merely guides for decision making. The source selection authority is responsible for 
independently determining whether noncost advantages are worth the cost/price that might be 
associated with a higher rated proposal. The decisive element is not the difference in ratings, but 
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the source selection authority’s rational judgement of the significance of that difference, based 
on an integrated comparative assessment of proposals. 


There is no magic formula for 
making the cost/technical tradeoff 


When inainng the cost/technical tradeoff leading to the selection decision, there is no “magic” 
formula. The cost/technical tradeoff and the source selection decision, which must be consistent 
with the solicitation, require that the source selection authority exercise reasonable business 
judgment in selecting the offeror for contract award. The information considered should include 
an analysis of the following: 

• The proposals’ total evaluated price or cost. 

• The significance of the differences in the noncost ratings as indicated by each 
proposal’s strengths, weaknesses, and risks. The strengths, weaknesses, and risks for 
each factor must be considered in light of the relative importance of each factor 
stated in the solicitation. 

In performing a tradeoff, consider following steps such as these to arrive at a rationale decision 
that can be well documented: 

• Compare the proposal differences that surfaced during your evaluations; 

• Define these differences and analyze their impact on performance objectives; 

• Make paired comparisons, comparing each proposal to each of the others; 

• Assess the best mix of cost and noncost benefits and determine whether the strengths 
of higher rated proposals are worth the price premium. 


A price premium must be justified 
regardless of the superiority of the rating 


It is essential to document cost/technical tradeoff judgments with detailed narrative explaining 
the relevant facts and supporting rationale. Mere statements of conclusion based on ratings or 
scores alone are not acceptable. The cost/technical tradeoff documentation mxist explicitly 
justify a price premium regardless of the superiority of the selected proposal’s technical or 
noncost rating. This justification is required even when the solicitation indicates that noncost 
factors are more important than cost/price. The justification must clearly state what benefits or 
advantages the Government is getting for the added cost/price and why it is in the Government’s 
interest to expend the additional funds. 
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Where it is determined that the noncost benefits offered by the higher priced, technically superior 
offeror are not worth the price premium, an explicit justification is also necessary. In this case, 
the documentation must clearly show why it is reasonable in light of the sigmficance of the 
differences to pay less money for a proposal of lesser technical merit. 

To determine which proposal provides the best value, the source selection authority must analyze 
the differences between competing proposals. This analysis must be based on the facts and 
circumstances of each acquisition and must be consistent with the solicitation. 

This analysis ensures a disciplined and documented process for an integrated comparison of 
proposals and a rational basis for the source selection authority’s ultimate decision. 

Documenting the Proposal Comparison 

Documentation explaining the final results of the evaluation should be prepared for the source 
selection authority to use in making the selection decision. This documentation should include 
the technical and/or past performance evaluation results, the cost/price evaluation, and the 
comparative value analysis, if applicable, for each proposal in the competitive range. The 
documentation should also include other considerations such as the results of negotiations. 

For more complex soiuce selections, this is accomplished by means of a formal report that is 
provided to the source selection authority. For less complex source selections, the 
documentation may be included as part of the Price Negotiation Memorandum. It should be 
simple but concise and should cross-reference rather than repeat information in existing 
documents as much as possible (e.g., the source selection plan, evaluation team consensus 
report). The analysis and comparisons in this documentation should be used as an aid to the 
source selection authority’s judgment -- not as a substitute for judgment. 

The documentation may contain: 

• Introductory information such as: 

• Data about the soxuce selection plan. 

• The basis for award and evaluation factors and subfactors. 

• Participants in the evaluation process. 

• SoUcitation requirements. 

• The number of offerors solicited. 

• The offerors who responded and those in the competitive range. 

• A summary of each proposal within the competitive range 

Summary matrix shows how evaluation results for each offeror are integrated at the 
factor level for use in documenting the proposal comparison. All ratings must be 
supported with narrative that describes the proposal’s strengths, weaknesses, and risks. 

FACTORS 
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TECHNICAL MERIT 
Summarizes assessment of the offeror’s 
proposal as measured against the technical 
subfactors 

Example: “Good” 

PROPOSAL MSK 

Summarizes assessment of the weaknesses 
and risks associated with the offeror’s 
proposed approach derived from the 
technical evaluation 
Example: Low Risk 

PERFORMANCE RISK 


COST/PRICE 

Reflects the total proposed cost or price. 
Where cost realism is evaluated, the cost 
also reflects the probable cost resulting 
from any adjustments made for cost 
realism. 

Example: Proposed Cost $XXX 
MostP Most Probable Cost $XXX 


Blueprint: Individual Offeror Evaluation Results 

• Comparative analyses of both cost and noncost factors of the proposals within the 
competitive range. The factors and subfactors evaluated should be discussed, first 
individually and then comparatively. The comparative cost analysis should explain 
the reasonableness, realism, and rationale of each offeror’s price or cost proposal. 
Each proposal’s major strengths, weaknesses, risks, as well as the details and results 
of the tradeoff analysis should be included. 

Matrix shows summary comparative evaluation results for all offerors within the competitive 
range. The supporting comparative analysis must document the integrated assessment of the 
technical (merit and risk), performance risk, and evaluated cost of the proposals relative to the 
factors and subfactors and to each other. 

TECHNICAL PERFORMANCE EVALUATED COST 


Summarizes assessment of the offeror’s 
demonstrated performance on recent, 
relevant contracts relative to the past 
performance subfactors. 
Example: Moderate Risk 
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OFFEROR 

TECHNICAL MERIT 

PROPOSAL RISK 

RISK 

(Most Probable Cost) 

A 

Excellent 

Low 

Low 

$171,503,971 

B 

Excellent 

Moderate 

Moderate 

$134,983,305 

C 

Good 

Low 

Moderate 

$120,976,836 

D 

Excellent 

High 

Moderate 

$150,840,308 

E 

Satisfactory 

Moderate 

Low 

$115,751,933 


Blueprint; Summary Comparison Matrix 

• A discussion of the overall impact of significant risks associated with each proposal 
within the competitive range. This discussion may address, for example: 

• Technical risks inherent in the offeror’s proposed approach. 

• Degree of confidence in the realism of the offeror’s cost or price proposal taking into 
consideration technical and schedule risk. 

• Production risks relating to new technologies and overall production competence. 

• Performance risks relative to the offeror’s record of recent and relevant past 
performance. 

• A summary of the comparative analyses, expressed in brief statements, of the issues 
considered significant to the source selection authority’s decision. If requested by 
the source selection authority, a selection recommendation would be included. 

Documenting the Selection Decision and Awarding the Contract 
Source Selection Decision Document for (specify product/service & RFP #) 

1. Decision Statement 

Example: As Source Selection Authority for this acquisition, I have determined that the XYZ 
product/service proposed by Offeror C provides the best overall value to satisfy Army needs. 
This selection was made based upon the factors and subfactors established in the solicitation and 
my integrated assessment and comparison of the strengths, weaknesses, and risks of the 
proposals submitted in response to the sohcitation. This memorandum documents the basis for 
my decision. 

2. Brief description of the product^service called for in the solicitation. 

3. Brief description of the Basis for Award including the major factors against which 


156 





proposals were measured and their relative order of importance. 

4. A list of offerors in the competitive range. 

5. Rationale for business judgments and tradeoffs. Include the following: 

• Succinctly compare each proposal to each of the others, focusing on key proposal differences 
(strengths, weaknesses, and risks) that surfaced in the evaluation and their impact on the 
acquisition objectives. 

• Clearly explain specific tradeoffs that led to the decision. 

• Clearly explain the specific benefits of technically superior offeror(s) and why they are or are 
not significant enough to warrant any additional cost. 

6. Summary. 

Example: In summary, based on my integrated assessment of all proposals in accordance with 
the specified evaluation factors and subfactors, it is my decision that Offeror C’s proposal offers 
the best overall value. 

Signature 

Source Selection Authority 


Blueprint: Typical Source Selection Decision Document 

Documentation setting forth the decision rationale must be prepared to support the source 
selection authority’s decision. The selection statement must be a stand-alone document that 
succinctly and accurately provides rationale for the selection. It should explain how the 
successful proposal measured up against other offerors based on the evaluation factors and 
subfactors in the solicitation. It should also explain the tradeoff judgments, including benefits 
associated with additional cost. 

This document becomes part of the official contract file and can even be released, provided that 
any inf ormation exempt under the Freedom of Information Act (FOIA) is not released. This can 
ease the debriefing process by showing offerors who request a debriefing the rationale and logic 
used by the source selection authority. After the source selection authority has signed the 
selection decision document, the contracting officer may execute and distribute the contract. 
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Notification and Debriefing of Unsuccessful Offerors 

When a contract is awarded as the result of a source selection, unsuccessful offerors must be 
debriefed and furnished the basis for the selection decision and contract award upon their written 
request. In addition, offerors excluded from the competitive range or otherwise excluded from 
the competition before award may request either a preaward or postaward debriefing. A 
debriefing may also be provided to the successful offeror. 

Debrief offerors promptly, at their request, as to the basis 
for the selection decision. Candidly explain the results of 
the Governments evaluation of their proposal without 
making any point-by-point comparisons with the content 
of other proposals. 

It is extremely important to promptly notify and debrief an offeror. Since each offeror puts 
considerable resources into preparing and submitting a proposal, fairness dictates that you 
explain why a proposal was unsuccessful. Early notification will also permit unsuccessful 
offerors to release the resources that would have been devoted to the contract effort so they can 
be used on other work. It is also in the Gtovemment’s best interest to fully inform the offeror of 
the proposal’s shortcomings so that the same mistakes are not repeated in future acquisitions. 
These actions reduce the cost of the competitive process and encourage the offeror to view the 
Government marketplace as a worthwhile area to invest its resources, thereby increasing 
competition. See Appendix F for guidelines on conducting debriefings. 
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Top Ten Messages 

1. Use a source selection approach commensurate with the acquisition’s evaluation needs. 

• Don’t make source selection more complicated and expensive than necessary. 

2. Invest in command or program resources neededfor a competent and well documented 
best value source selection. 

• Include the source selection authority as an active participant — this is not a 
*yigure head** position. 

• Train evaluators in best practices. 

3. Understand the importance of planning. 

• Think through the entire process. 

• Prepare a source selection plan before the solicitation. 

4. Structure the solicitation to communicate effectively to potential offerors: 

• Government requirements and mission objectives. 

• The information needed to evaluate the proposal 

• The ground rules the government will use to select the best value proposal 

• Evaluation factors and subfactors and their relative importance. 

5. State requirements functionally to the maximum extent possible. 

• Limit use of design requirements. 

6. Document strengths, weaknesses, risks and associated value of proposals to support the 
cost/technical tradeoff. 

• Recognize that numerical rating systems can imply false precision and limit 
flexibility in the decision making process. 

7. Justify a price premium with cost/technical tradeoff documentation regardless of the 
selectedproposaVs technical superiority. 

8. Ensure that the source selection decision is consistent with the solicitation. 

9. Do a proper and timely debriefing. 

10. Document lessons learned. 


The proponent of this pamphlet is the United States Army 
Materiel Command (AMC). Please send comments and 
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suggested improvements to U.S. Army Materiel Command, 
Attn: AMCRDA-AP, 5001 Eisenhower Avenue, Alexandria, 
VA 22333-0001 or submit your comments via the Internet 
at the following address: http://www.army- 
acquisition.net/amcweb/fr_ssll.htm. 


For the Commander: 
Official: 


James M. Link 
Major General USA 
Chief of Staff 

Leroy Tillery 

Chief, Printing and Publications Branch 
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Definitions 


Competitive Range. All proposals that the contracting officer determines to be the most highly 
rated based on the cost and noncost evaluation factors stated in the solicitation. 

Design Specification. A specification that establishes precise measurement, tolerances, 

materials, in process and finished product tests, quality control, inspection requirements, 
and other specific details of the deliverable. 

Evaluation Factors and Subfactors. Descriptions of those aspects of a proposal that will be 
evaluated quantitatively or qualitatively to assess which proposal can best meet the 
Government’s requirements as described in the solicitation. 

Evaluation Standards. A baseline level of merit or acceptability used for measuring how well 
or whether an offeror’s response meets the solicitation’s requirements with respect to an 
evaluation factor or subfactor. 

Functional Specification. A specification that describes the deliverable in terms of form, fit, 
and function and performance characteristics to satisfy the intended use. 

Performance Specification. A specification that states requirements in terms of the required 
results and provides criteria for verifying compliance, without stating methods for 
achieving results. It defines the functional requirements, the operational environment, and 
the interface and interchangeability requirements. 

Rating. The application of a scale of words, colors, or niunbers, used in conjunction with 
narrative, to denote the degree to which the proposal has met the standard for a noncost 
factor. 

Source Selection/Evaluation Plan. The document that explains how proposals are to be 

solicited and evaluated to make the selection decision. It is the Government’s plan for how 
it intends to acquire its needs. 

Source Selection Process. The process of soliciting and evaluating proposals for award in a 
competitively negotiated environment. 

Specification. A description of the technical requirements for a material, product, or service to 
be provided under a contract that includes the criteria for determining whether the 
requirements are met. 

Statement of Work (SOW). The complete description of work to be performed under the 
contract, encompassing all specifications and standards established or referenced in the 
contract. 

Work Breakdown Structure. A basic firamework, similar to a table of contents, which outlines, 
divides, and subdivides to successively lower levels, the government’s requirements as set 
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forth in the SOW. Permits a logical arrangement of the elements of the SOW and a tracing 
of work effort and costs proposed by each offeror under each of the elements. 
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Guidelines for Evaluating Past Performance in Source Selection 


Overview 


This appendix is designed to help you participate in the evaluation of past performance when 
using the tradeoff process in source selection. It should always be used in conjunction with the 
basic best value guide. 


Introduction 


An offeror’s performance record is an important consideration in government source selections. 
In our private lives, we make source selections every day. This can be as mundane as selecting 
the brand of toothpaste we use or one that represents a much greater portion of our earnings such 
as a new car. If we can consider how a product worked (or didn’t work) for us in the past, get 
advice from our friends, read Consumer Reports, and check the contractor out with the Better 
Business Bureau, wouldn’t it make sense to give the same credence to past performance when 
spending taxpayer dollars? 

Using the contractor’s past performance as a significant evaluation factor that will be traded off 
with cost/price and other noncost factors will benefit us in at least two ways. 

First, it is virtually impossible for an offeror to convey, in the proposal, how it will actually 
perform the requirement and what the actual costs will be (not just the instant contract costs, but 
the total life cycle costs associated with obtaining the system, supply, or service). A better tool 
for predicting how an offeror will perform in the future is to examine its past track record on 
contracts for similar requirements of the same scope and complexity. 

Second, it permits us to recognize the good performer. In the commercial market, poor 
performers are weeded out as consumers select vendors that live up to the consumer’s 
expectations. Those contractors that provide what is perceived as the best value in supplies or 
services are the ones that survive. A contractor that delivers what the contract requires without 
extensive follow-up effort on our part is clearly delivering better value than a contractor that 
charges the same price, yet needs constant surveillance by our personnel to ensure performance. 
It also shifts the emphasis from writing the best proposal to performing the best work — 
performing as promised. 


Role of Past Performance in Source Selection 


Proposal Risk vs. Performance Risk 

It is important to differentiate between two types of risk usually evaluated in a source selection. 

Proposal risks are those associated with an offeror’s proposed approach in meeting the 
requirements of the solicitation. Proposal risk summarizes the risk derived from the techmcal 
evaluation. It is an overall assessment driven by each of the subfactors within the technical 
factor. 
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Performance risks are those associated with an offeror’s likelihood of success in performing the 
solicitation’s requirements as indicated by that offeror’s record of current or past performance. 
Performance risk is assessed separately and assigned a narrative rating. Both proposal and 
performance risks are, in accordance with the basic guide, integrated with the other evaluation 
results when presented to the source selection authority for a final decision. 


Performance risks reflect an offeror’s 
likelihood of success based on 
their current or past work record 


Past Performance vs. Responsibility Determinations 

It is also important to distinguish comparative past performance evaluations used in the tradeoff 
process from pass/fail responsibility determinations. 

Preaward surveys and pass/fail evaluations in the low price techmcally acceptable process help 
you determine whether an offeror is responsible. Responsibility is a broad concept that 
addresses whether an offeror has the capability to perform a particular contract based upon an 
analysis of many areas including financial resources, operational controls, technical skills, 
q uali ty assurance, and past performance. These surveys and evaluations provide a “yes/no,” 
“pass/fail,” or “go/no-go” answer to the question, “Can the offeror do the work?” to help you 
determine whether the offeror is responsible. 

Unlike a pass/fail responsibility determination, a comparative past performance evaluation 
conducted using the tradeoff process is a very specific endeavor that seeks to identify the degree 
of risk associated with each competing offeror. Rather than asking whether an offeror can do 
the work, you should ask, will it do that work successfully? In short, the evaluation describes the 
degree of confidence the government has in the offeror’s likelihood of success. If properly 
conducted, the comparative past performance evaluation and the responsibility determination 
will complement each other and provide you with a more complete picture of an offeror than 
either one could by itself. 


Past Performance vs. Experience 

Another important issue is the difference between contractors’ experience md their past 
performance. Experience reflects whether contractors have performed similar work before. Past 
performance, on the other hand, describes how well contractors performed the work. In other 
words, how well did they execute what was promised in the proposal/contract. Both of these 
areas are considered when making a responsibility determination. Either past performance or 
experience can be considered as source selection factors or subfactors, where they can either 
stand alone or be considered under performance risk. 

If experience and past performance are separate evaluation factors or subfactors, make certain 
that you clearly define the terms in the solicitation. This will help you avoid the potential for 
double counting by asking for the same information. Do not confuse evaluation of past 
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experience with evaluation of past performance. It is proper, however, to distinguish company 
experience from personnel experience and evaluate both. 

When to Evaluate Past Performance 

Past performance shall be included as an evaluation factor in competitively negotiated 
acquisitions unless the contracting officer determines that it is inappropriate and documents the 
rationale. 


Planning the Past Performance Evaluation 
Forming an Evaluation Group 

In complex acquisitions it may be necessary to form a formal group to specifically evaluate past 
performance. In smaller dollar value acquisitions that do not involve complex requirements, the 
evaluation may be accomplished with only one or two people to determine that same risk. This 
evaluation group may operate separately from the proposal evaluation team or may operate as a 
separate subgroup of that team. Note, however, that the past performance evaluation should be 
conducted independently of any other evaluation. 

Whether you choose to use an informal or formal evaluation group, the members should be 
experienced and trained. We will focus on the structure, composition and evaluation process of a 
formal evaluation group, but bear in mind that while the functions of informal evaluations are 
basically the same, they should be a lot less time consuming and less involved. 

Objectives of the Evaluation Group 

The evaluation group is responsible for conducting the past performance evaluation to determine 
the degree of risk involved in accepting an offeror’s promises of performance. This analysis 
results in a performance risk assessment. The evaluation group documents these performance 
risk assessments and identifies strengths and weakness in each offeror’s past performance. 

The group often uses the offeror’s proposal as the starting point -- with the proposal acting 
p rimar ily as a souTce of reference. Group members then use independent sources of information 
outside of the offerors’ proposals to determine how well those offerors performed in the past. 

Evaluation Group Membership and Training 

The membership and structure of your evaluation group should be tailored to each acquisition. 
Ideally, the membership should be multifunctional and include people who have contracting, 
cost/price, contract administration, supportability/logistics, and technical/program management 
expertise, as appropriate. Those whom you select should also have prior source selection 
experience and be capable of making soimd and impartial judgments. The group’s structure 
should enhance its ability to independently evaluate performance risk. 

As a best practice, it is important that contracting organizations set up their own training program 
for conducting source selections. Individual programs should include tools and techniques for 
obtaining and evaluating contractor performance data. 
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The heart of the performance risk assessment is the information gathering process. Through 
questionnaires, telephone interviews, and site visits, and by tapping existing data sources, the 
group can obtain a detailed and useful picture of an offeror’s past performance. Because of the 
importance of the information gathering process, it is absolutely critical that group members 
have the ability to conduct meaningful telephone interviews. They should also be able to 
agg iTTiTlatft data, exercise sound judgment, arrive at conclusions that make common sense, and 
co mmuni cate those conclusions effectively both orally and in writing. 

The size of the group should reflect the number of ofiferors expected to respond to the solicitation 
as well as the nature and complexity of the solicitation requirements. The best practice is to limit 
the size of the group to as small a number as is realistic for the specific circumstances of the 
acquisition. A group of at least two members of different functional disciplines enhances 
opportunities for dialogue, brainstorming, and in-depth fact finding. 

A plan for evaluating past performance should be developed early in the process and made a part 
of the source selection plan. 


What Subfactors Should Be Used? 

The past performance subfactors, if any, should be tailored to the specific circumstances of the 
acquisition, but need not mirror &ose of the proposal evaluation. In most cases the evaluation 
group should at least consider the offeror’s record for on time delivery, good technical quality, 
and cost control to determine its likelihood of success in performing the solicitation’s 
requirements. 

Some suggested subfactors include: 

• The offeror’s record of conformance to contract specifications and standards of good 
workmanship. This may include consideration of quality awards or certifications; 

• The offeror’s effectiveness in containing and forecasting costs on any previously 
performed cost reimbursable contracts; 

• The offeror’s adherence to contract schedules, including the administrative aspects of 
performance; 

• The offeror’s history for reasonable and cooperative behavior, commitment to 
customer satisfaction, timely award and management of subcontracts, and whether 
the offeror met any applicable goals for subcontracting with small entities. 

Data Relevancy 

Relevancy should not be described as a subfactor. Relevancy is a threshold question when 
considering past performance, not a separate element of past performance. Irrelevant past 
performance should not form the basis of a performance risk assessment. 

Although the group may consider data available firom any sources, its main sources of 
information are often the references cited by offerors in their proposals. Upon receipt of 
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proposals, you will determine which of the offeror’s past contract efforts relate to the solicitation 
requirements. The evaluation group should screen the information provided for each of the 
referenced contracts to make an initial determination of its relevance to the current requirement. 
However, the source selection authority is responsible for the final determination of relevancy. 

Relevancy is defined as something that has a logical connection with the matter under 
consideration. Such aspects of relevance include the type of effort (e.g., development, 
production, repair), and the type of requirement (e.g., weapon systems, information systems, 
engineering services, programmed depot maintenance). TTie objective of the screening is to 
remove fi'om consideration those contract references that are clearly xmrelated to the type of 
effort sought. Note that valuable information can be obtained firom seemingly unrelated prior 
contracts regarding considerations such as technical capability, management responsiveness, 
proactive process improvements, and ability to handle complex technical or management 
requirements. Other members of the source selection team may be consulted as necessary for 
assistance in determining relevancy. 


TaUor data relevancy and recency 
to the specifics of each acquisition 


In some cases, previous contracts as a whole may be similar to the current contract while in 
others only portions of previous contracts may be relevant. For example, we use Ada software 
language in many different government systems. If a solicitation calls for the development of 
Ada software for an aircraft system, the contractor might identify a previous effort where it 
developed Ada software for a satellite terminal. We may consider Aat previous effort to be 
relevant for pmposes of assessing the contractor’s ability to develop Ada software even though 
the underlying system is different from the current requirement. Another example is the 
evaluation of the contractor’s management, planning, and scheduling of subcontractors on a past 
service contract for a current production requirement calling for integration skills. 

Note that, in the case of mergers or joint ventures, only that part of the newly formed corporation 
identified with the relevant experience should be evaluated. 

The evaluation group should consider the most recent data available. The best practice is to 
select similar efforts that are either still in progress or just completed, and that have at least one 
year of performance history. While the actual cut-off time should be determined by the 
contracting officer on a case-by-case basis, the currency of the information requested should be 
determined by the commodity and the specific circumstances of the acquisition. 

How Much Weight to Give Past Performance 

Past performance should be given sufficient evaluation weight to ensure that it is meaningfully 
considered throughout the source selection process and will be a valid discriminator among the 
proposals received. 


Weight past performance sufficiently 
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to ensure it is a valid discriminator 


What are the Rating Categories? 

In planning the acquisition, the evaluation group develops a rating scheme for evaluating past 
performance. The group may use the following definitions of performance risk to describe the 
results of its assessment: 

• PoorA^ery High Performance Risk. Based on the offeror’s performance record, 
extreme doubt exists that the offeror will successfully perform the required effort. 

• Marginal/High Performance Risk. Based on the offeror’s performance record, 
substantial doubt exists that the offeror will successfully perform the required effort. 

• Adequate/Moderate Performance Risk. Based on the offeror’s performance record, 
some doubt exists that the offeror will successfully perform the required effort. 

• Good/Low Performance Risk. Based on the offeror’s performance record, little 
doubt exists that the offeror will successfully perform the required effort. 

• Excellent/Very Low Performance Risk. Based on the offeror’s performance record, 
essentially no doubt exists that the offeror will successfully perform the required 
effort. 

• Unknown Performance Risk. No performance record identifiable. See “How to 
Evaluate No Past Performance.” 

How to Evaluate No Past Performance 

In most cases the evaluation group will find some related government or other public or private 
past performance information for each contractor and subcontractor. Such information will 
usually surface if the evaluation approach allows a broad interpretation of relevancy or takes into 
accoimt information regarding the past performance of predecessor companies, key personnel 
who have relevant experience, or subcontractors that will perform key aspects of the 
requirement. 

Occasionally, however, an evaluation group cannot find any relevant information. In those 
cases, you must treat an offeror’s lack of past performance as an unknown performance risk, 
having no positive or negative evaluative sigmficance. This allows the government to evaluate 
past performance in a manner that is fair to newcomers. It neither rewards nor penalizes firms 
without relevant performance history. The method and criteria for evaluating offerors with no 
relevant past performance information should be constructed for each specific acquisition to 
ensure that such offerors are not evaluated favorably or unfavorably on past performance. 

You may use a variety of rating methods to evaluate offerors with no past performance histoiy. 
Regardless of the method selected, the solicitation must clearly describe the approach that will be 
used for evaluating offerors with no relevant performance history. 
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Encourage newcomers to identify other 
related past performance information 


You can also ease the impact on newcomers by including language in the solicitation that 
encourages them to identify related past performance information for Federal, state and local 
government contracts, commercial contracts, key personnel, and major subcontractors. 

What to Include in the Solicitation 


Include this form with the solicitation’s instractions to offerors to simplify the submission and 
evaluation of past performance information 

(To be completed by the offeror) 

1. Contract Number: __ 

2. Contractor (Name, Address and Zip Code): 

3. Type of Contract: Negotiated_Sealed Bid_ 

Fixed Price_ Cost Reimbursement_ Hybrid (explain)_ 

4. Complexity of Work: Difficult_ Routine_ 

5. Description, location & relevancy of work: 

6. Contract Dollar Value: _ 

Status: Active_ Complete_ 

7. Date of Award: _ 

Contract Completion Date (including extensions): _ 

8. Type and Extent of Subcontracting: 
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9. Name, Address, Tel. No. & e-mail of the Procuring Contracting Officer and/or the 
Contracting Officer’s Representative (COR) (and other references, e.g.. Administrative 
Contracting Officer, if applicable): 


Blueprint: Format for Sabmission of Past Performance Information 

The solicitatipn, at a mi nimum , must clearly describe the approach you will use to evaluate past 
performance. The approach must include what past performance information you will evaluate, 
how it will be evaluated, its weight or relative importance to the other evaluation factors and 
subfactors, and how you will evaluate offerors with no past performance history. The amount of 
information you request should be tailored to the circumstances of the acquisition. The amount 
should be reasonable and not impose excessive burdens on offerors or evaluators. The proposal 
evaluation inf ormation, as a minimum, should clearly state that: 

• The government will conduct a performance risk assessment based upon the past 
performance of the offerors and their proposed major subcontractors as it relates to 
the probability of successfully performing the solicitation requirements; 

• In conducting the performance risk assessment, the government may use data 
provided by the offeror and data obtained fi'om other sources; and 

• While the government may elect to consider data obtained fi-om other sources, the 
burden of providing thorough and complete past performance information rests with 
the offeror. 

The proposal submission instructions must, as a minimum, instruct offerors to submit recent and 
relevant information concerning contracts and subcontracts (including Federal, State, and local 
government and private) that demonstrate their ability to perform the proposed effort. 

Rather than having the offeror submit a lengthy narrative response, the information requested 
should be limited to a half-page sununary of the offeror’s performance claim for each contract or 
subcontract. The summary should include contract numbers, contract type, description and 
relevancy of the work, dollar value, contract award and completion dates, and names, phone 
numbers, and e-mail addresses for references in contracting and technical areas. 

In addition, offerors should be given the opportunity to explain why they consider the contracts 
they have referenced as being relevant to ffie proposed acquisition. Relevancy is defined as 
something that has a logical connection with the matter under consideration. (See section 
entitled. Data Relevancy.) The instructions should also permit offerors to provide information 
on problems encountered on such contracts and the actions taken to correct such problems. Also, 
it is important that the offeror specifically describe the work that its major subcontractors will 
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long. 

• If a problem surfaced, ask what the government and contractor did to fix it. 


• Ask for a description of the types of personnel (skill and expertise) the contractor used and the 
overall quality of the contractor’s team. Did the company appear to use personnel with the 
appropriate skills and expertise? 

• Ask how the contractor performed considering technical performance or quality of the product 
or service; schedule; cost control (if applicable); business relations; and management. 

• Ask whether the contractor was cooperative in resolving issues. 

• Inquire whether there were any particularly significant risks involved in performance of the 
effort. 

• Ask if the company appeared to apply sufficient resources (personnel and facilities) to the 
effort. 

• If the company used subcontractors, ask: What was the relationship between the prime and 
subcontractors? What was the management role of the prime and how well did it manage the 
subcontractors? Did the subcontractors perform the bulk of the effort or just add breadth or 
depth on particular technical areas? If the subcontractors worked on specific technical areas, 
what were those areas and why were they accomplished by the subcontractors rather than the 
prime? 

• If a problem is imcovered that the reference is unfamiliar with, ask for another individual who 
might have the information. 

• Ask if this firm has performed other past efforts with the reference’s agency. 

• Ask about the company’s weak points or what the reference liked least. 

• Inquire whether the reference has any reservations about recommending a future contract 
award to this company. 

• Inquire whether the reference knows of anyone else who might have past performance 
information on the offeror. 


Blueprint; Typical Past Performance Questions 

Where to Conduct Telephone Interviews 

Following the screening of previous contracts for further in-depth review, each evaluation group 
member should send questionnaires and/or initiate telephone calls to the identified references for 
those efforts. The interviewing and reporting of results are usually individual efforts conducted 
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by each evaluation group member. However, it is sometimes helpful to collect information as a 
group through the use of conference calls. In any event, the environment in which this work is 
done significantly impacts both the time required to complete this portion of the process and the 
quality of the results. These activities are hampered severely if each group member attempts to 
conduct telephone interviews at their normal work site with all of its attendant interruptions, 
distractions, and security risks. 

If, on the other hand, the group members are able to assemble as a group for telephone 
interviews, they will be able to provide considerable reinforcement and instant feedback for one 
another. Group members should be able to devote their undivided attention to this initial 
assessment process. Although this approach requires a secure area that is large enough to 
accommodate all of the group members, the resulting benefits are significant. 

How to Conduct Telephone Interviews 

The telephone interview process is an art form. Until a smooth conversation pattern is 
developed, it is an inherently uncomfortable situation for many people. There will be some 
difficulty learning how to start a telephone interview, keep it moving, and cover all important 
areas. As the interviewing process continues, the evaluation group member usually uncovers 
special items of interest that he or she will want to pursue through follow-up calls. 

At least two references should be contacted on each previous contract effort selected for in-depth 
review. The contracting officer, program manager, and contracting officer’s r^resentative often 
prove to be excellent sources of information. Additional references are often identified during 
the interviews. Maximum effectiveness occurs when the expertise of tiie evaluation group 
interviewer matches that of the reference. 

Prior to initiating a telephone interview, a group member should gather all available infonnation 
on a specific effort and draft a list of questions. There may be a common group of questions for 
all offerors and/or tailored questions for each offeror, depending upon the circumstances. These 
questions can either be sent as questionnaires to each reference or be used by the group member 
during the telephone interview. 

At the start of each telephone interview, the group member should explain the purpose of the call 
and request voluntary assistance fi-om the reference. The interviewer should explain that he or 
she will document the results of the conversation and send a copy of the memorandum to the 
reference for verification. There is usually no need to divulge the solicitation number, program 
description, or other identifying information to the reference. If you do so, you need to obtain a 
nondisclosure statement. 

In most instances the reference will willingly provide the information requested. In those rare 
cases when the reference is reluctant to participate, the interviewer should ^sure the reference of 
anonymity. At the least, the reference should be requested to provide additional references. 

It is important to pursue and document the underlying facts supporting any conclusionaiy 
statements received on a contractor. The evaluation group member can determine neither the 
magnitude of a reported problem nor its possible impact on the current risk assessment without 
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receive the greatest consideration. The assessment should include a description of the vmderlying 
rationale fiar the conclusions reached. As long as that rationale is reasonable, it will withstand 
scrutiny even if other reasonable conclusions exist. 

A word of caution is appropriate concerning offeror promises to correct past performance 
failures, as opposed to actions already taken to correct such failures. A promise to improve does 
not, by itself, improve past performance. However, demonstrated corrective actions reflect a 
commitment to rectify past performance problems, and therefore, can lower the risk of similar 
performance failures. 

Disclosing Past Performance Information to Offerors 

The contracting officer must also provide offerors with the opportunity to comment on negative 
past performance information on which offerors have not had a previous opportumty to 
comment. This practice ensures fairness for the competing offerors. The validation process is 
particularly important when the negative information is provided by only one reference, or when 
there is any doubt concerning the accuracy of the information. Usually, negative information 
reflects performance that was less than satisfactory, although this is a judgment call that will 
depend upon the circumstances of the acquisition. Note that while the government must disclose 
past performance problems to offerors it shall not disclose the names of individuals who 
provided information about an offeror’s past performance. 

A special problem arises with respect to subcontractors. Past performance information 
pertaining to a subcontractor cannot be disclosed to a private party without the subcontractor’s 
consent. Because a prime contractor is a private party, the government needs to obtain the 
subcontractor’s consent before disclosing its past performance information to the prime during 
negotiations. There are a variety of ways to obtain subcontractor consent. For example, the 
solicitation could require the prime to submit its subcontractor’s consent along with flie prime’s 
proposal to the government. 

What to Include in the Assessment Report 

The goal is to avoid saying too much or too little in the evaluation report. Although there is no 
need to restate everjdhing contained in the telephone memoranda, the evaluation group must 
provide the source selection authority with sufficient information to make informed judgments. 

Conclusionary statements must be supported by the imderlying factual basis. The best practice is 
to state the conclusion and provide specific examples that support that conclusion. 

To ensure that the risk assessments provide the necessary background information and are 
structured consistently, the entire evaluation group should review and evaluate the report on each 
offeror. During this review, the evaluation group should correct statements that appear 
unsupported, inconsistent, or urmecessary. 

Occasionally the evaluation group will be unable to arrive at a unanimous agreement on a 
particular risk assessment. If this occurs, the evaluation group may include the dissenting 
opinion as part of the assessment report. 
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Reporting the Past Performance Evaluation Results 

The evaluation group’s submission of the assessment report usually completes the major portion 
of its work. The evaluation group leader, should remind the source selection official of the 
pmpose of the group and the past performance evaluation approach, including offerors with no 
past performance history, described in the solicitation. This is to ensure that everyone fully 
comprehends the sigmficance of the results being reported. Experience reveals that source 
selection officials are more apt to rely upon evaluation group results if they thoroughly 
understand the process. 


How to Treat Past Performance Information 

Inf ormation concerning the past performance of an offeror or of its proposed subcontractors 
should be treated as sensitive source selection information. This information sometimes includes 
information that is proprietary, such as trade secrets and confidential commercial or financial 
that would not be released under the Freedom of Information Act. Current laws, 
regulations, and policies governing storage, access, disclosure, and marking of source selection 
and proprietary inf ormation must be observed at all times. Questions concerning the procedures 
for the handling of past performance information should be referred to the contracting officer or 
legal counsel for resolution. 

The evaluation group must retain the records of its activity throughout the source selection 
process. Upon contract award or cancellation of the solicitation, all evaluation group records are 
provided to the contracting ofScer for retention along with the other source selection documents. 

Improving the Evaluation Process 

The methods described in this guide will evolve as our needs change and as our knowledge base 
expands. It is important for all of us to share lessons learned in this process. You can help 
others benefit fi-om your experiences by submitting lessons learned via the Internet at 
http://www.army-acquisition.net/amcweb/ssp/ssp_ap_d.htm#57 
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Guidelines for Conducting Debriefings 


Overview 


This appendix provides easy to follow guidance for conducting both preaward and postaward 
debriefings. It should always be used in conjunction with the basic best value guide. Using this 
guidance will result in meaningful debriefings and will instill greater confidence in the 
acquisition process. Comprehensive and open debriefings will strengthen and enhance our 
relationship with industry. 


What Is A Debriefing? 

A debriefing is a meeting between government personnel and an offeror who has been eliminated 
from the competition either prior to or after contract award. The purposes of a debriefing are: 

• To explain the rationale for exclusion from the competition. 

• To instill confidence in the offeror that it was treated fairly. 

• To assure the offeror that proposals were evaluated in accordance with the 
solicitation and applicable laws and regulations. 

• To identify weaknesses in the offeror’s proposal so the offeror can prepare better 
proposals in future government acqmsitions. 

• To reduce misunderstandings and protests. 

• To give the offeror an opportunity to provide feedback regarding the solicitation, 
discussions, evaluation, and the source selection process. 

A debriefing is NOT: 

• A page-by-page analysis of the offeror’s proposal. 

• A comprehensive point-by-point comparison of the proposals of the debriefed 
offeror and the successful offeror(s). 

• A debate or defense of the government’s award decision or evaluation results. 


Debriefings enhance nnderstanding 
of the selection process 


Who Is Entitled To A Debriefing? 
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Whenever the Govemnient awards a contract based on competitive proposals, an unsuccessful 
offeror is entitled to a debriefing if it timely requests one. Debriefings are not required when 
sealed bidding procedures are used. 

Offerors excluded from the competitive range or otherwise excluded from the competition before 
award may request a preaward or postaward debriefing. 

Good business practice dictates that the Govermnent should debrief the awardee if requested. 
Although the Government often conducts a postaward conference with the awardee, that meeting 
does not necessarily substitute for a debriefing. 

When Must A Debriefing Be Conducted? 

The Government must conduct a debriefing for an unsuccessful offeror if: 

(1) the offeror makes a written request for a debriefing, and 

(2) the request is received by the contracting activity within 3 days after the offeror 
received notice of exclusion from the competition or contract award. 

Here*s a simple rule for computing time periods: Count days as 
calendar days and include weekends and legal holidays. Don’t 
count the day the offeror received the notice. Start with the next day. 

For example, if the offeror received the notice on Tuesday, the 1st 
day is Wednesday, the 2d day is Thursday, and the 3d day is Friday. 

Thus, you must receive the request for the debriefing no later than 
close of business on Friday. 

Establishing the date the offeror received the notice may be difficult if the notice is sent by 
regular mail. Accordingly, you should consider sending the notice by mail with return receipt 
requested or by electronic transmission (fax) with immediate acknowledgment requested. 
Remember, every day of delay in notifying the offeror usually extends by one day the time in 
which a protest may be filed. 

If the request is for a preaward debriefing, the contracting officer shall make every effort to 
debrief the imsuccessful offeror as soon as practicable. If there are compelling reasons that it is 
not in the Government’s best interest to conduct the debriefing prior to award, the contracting 
officer can delay it no later than the time when postaward debriefings are provided. In such 
cases, you must document the contract file with the rationale for the delay. This debriefing 
should then be conducted as a postaward debriefing. 

If an offeror, who has been eliminated from the competition prior to award requests a delayed 
debriefing (i.e., postaward), the date the offeror knew or should have known the basis of a protest 
shall be the date the offeror received notice of the exclusion fixjm the competition. 

If you receive a timely request for a postaward debriefing, you must conduct the debriefing 
within 5 days, to the ma ximum extent practicable, after receipt of the request. Remember that 
the scheduling of a debriefing may have significant ramifications if a protest is later filed. A 
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protester filing a protest with the General Accoimting Office is entitled to an automatic 
suspension of contract performance if the protest is filed within 5 days of the debriefing date 
offered to the protester (or 10 days after contract award, whichever is later). 

Thus, it is extremely important that you schedule the debriefing for the earliest possible date for 
the Government. You should inform the offeror of the scheduled date in writing by electronic 
means with immediate acknowledgment requested. If the offeror is imable to attend the 
scheduled date and requests a later date, the offeror should be required to acknowledge in writing 
that it was offered an earlier date, but requested the later date instead. This procedure will 
protect the Government’s interests if the offeror subsequently files a protest. 

If an offeror submits an untimely request for debriefing (i.e., received more than 3 days after 
notice of elimination from the competition or contract award), the contracting officer should 
nonetheless conduct a debriefing if feasible. However, the contracting officer should inform the 
offeror that the request is imtimely. Untimely requests do not invoke the statutory provision for 
automatic suspension of contract performance imless the protest is filed within 10 days of 
contract award. 

The time periods stated in this guidance were current on the date of publication. However, 
always check with your legal counsel for the latest General Accounting Office rules and 
revisions. 


Where Is A Debriefing Held? 

The contracting officer is responsible for selecting a suitable location for the debriefing that 
ensures a professional presentation in a nondistracting enviromnent. 

A conference room will normally suffice; Obviously, the room must be equipped with the 
necessary audiovisual equipment that will be used during the debriefing. The contracting officer 
should consult with the offeror to ensure that adequate seating will be available for all personnel 
attending. If classified material will be discussed, appropriate arrangements must be made. 

Although face-to-face debriefings are frequently preferable, it is appropriate to conduct 
debriefings by telephone or other electronic means. Sometimes, it will be financially prohibitive 
for the offeror to attend in person. The needs of the offeror should be afforded due 
consideration, but the contracting officer makes the final decision as to the debriefing location. 

If some of the government personnel are located at an installation other than the contracting 
office, the contracting officer may decide that those individuals’ attendance is best accomplished 
by telephone or video conference. The possibilities are endless — make use of available 
technology and do what makes sense! 

Who Attends A Debriefing? 


Government Personnel 


The contracting officer is responsible for chairing the debriefing. This doesn’t mean that the 
contracting officer must conduct the entire debriefing. In fact, usually the contracting officer 
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will rely on government technical and cost/price personnel to present the portions of the 
debriefing that address those specialized areas of the offeror’s proposal. The debriefing should 
be conducted with only one offeror at a time. 

The contracting officer identifies the Government debriefing team members. The selection is 
based on the complexities presented in each acquisition. The key is to ensure that 
knowledgeable Government personnel are present. Because of the statutory requirement for a 
prompt debriefing, the contracting officer should tentatively select the team before the contract 
award is announced. 

When determining the composition of the team, you must keep in mind the objectives of a 
meaningful debriefing. Above all, the Government must display that it fully understood the 
offeror’s proposal. If this is not conveyed, the offeror will obviously have little confidence in the 
conduct of the acquisition. Many debriefings have failed merely because the contracting officer 
did not ensure appropriate Government personnel were present. 


Involve government legal counsel in debriefings 


The contracting officer’s legal counsel should attend the debriefing. Legal counsel should also 
assist in the preparation of the debriefing. If the acquisition is in litigation or under protest, legal 
counsel must attend the debriefing. Legal counsel should be informed of any indicators that a 
protest is likely. However, the contracting officer will not deny a debriefing because a protest is 
threatened or has already been filed. 

Debriefed Offeror Personnel 

The contracting officer should ask an offeror scheduled for a debriefing to identify all individuals 
by name and position who will attend the debriefing. Normally, no limitation shoiild be placed 
on the personnel the debriefed offeror may bring to a debriefing. Nonetheless, space limitations 
of Government facilities may require restrictions in extraordinary cases on the number of offeror 
personnel invited to attend. The contracting officer should not unpose restrictions unless the 
contracting officer has determined that all suitable alternate facilities are unavailable. 

There may be times when you shoiild consider offering to have high level officials firom the 
requiring activity present at the debriefing if the offeror in turn agrees to have commensurate 
management officials present. This has proven effective when high level officials were 
substantially involved in the acquisition or the acquisition received extensive high-level visibility 
or scrutiny. 


Preparing For A Debriefing 


A debriefing requires preparation! 
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Only the foolhardy will attempt to conduct a debriefing unprepared. Experience has shown that 
going into a debriefing imprepared is the surest way to lose the confidence of the offeror and 
increase the prospects of a protest. 

Because debriefings are time sensitive, preparation must begin before proposal evaluation is 
complete. Usually, the proposal evaluation board will assist you in preparing debriefing charts 
and conducting the debriefing. Accordingly, at the time the evaluation board is formed, you 
must inform the evaluators that their duties include assisting with debriefings. 

Preparation for the debriefing should be thorough. Debriefings permit offerors to learn their 
strengths and weaknesses and how to improve future Government proposals. Offerors may also 
rely, however, on these sessions to influence their decision regarding filing a protest. An 
effective debriefing can often deter a protest by demonstrating that the Government conducted a 
thorough, fair evaluation and made a sound decision according to the established solicitation 
evaluation methodology. 

The extent of preparation necessary varies considerably with the complexity of each acquisition. 
Sometimes, merely preparing debriefing charts is sufficient. Other times, dry run rehearsals are 
necessary. 

Finally, all government personnel attending the debriefing must be briefed on their roles and 
expected demeanor during the debriefing. Argumentative or overly defensive conduct should be 
discouraged, and Government personnel should be instructed to make a positive presentation. 

What Materials Should Be Brought Into the Debriefing Room? 

Debriefing material normally consists of briefing charts and notes prepared for use during the 
debriefing. 

• Experience has shown that Government persoimel should Not bring the following 
materials into the debriefing room: 

• Proposals of other offerors, and 

• Proposal evaluation reports of other offerors’ proposals. 

The presence of these documents can lead to the inadvertent disclosure of proprietary 
information and prohibited point-by-point comparisons of proposals. 

Experience has also shown that a faulty memory or misstatements by Government personnel are 
detrimental to a successful debriefing. Thus, contracting officers must ensure that necessary 
notes or other documents are accessible during the debriefing. 

Debriefing the Unsuccessful Offeror 
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SOURCE SELECnON “BLUEPRINT” 

Typical Unsuccessful Offeror Debriefing Charts - Post Award 


High Performance Computer Contract 

DAAX08-XX-C-1234 



Unsuccessful Coip.,Inc. 
Debriefing 
June X XXXX 


Typical Debriefing Chart No. 1 - Cover Page 




SOURCE SELECTION “BLUEPRINT” 

Tjppical Unsuccessful Offeror Debriefing Charts - Post Award 

AGENDA 

• Introduction 

• Ground Rules 

• Source Selection Process 

• Evaluation Criteria 

• Rationale for Award Decision 

• Overall Ranking of Proposals 

• Relevant Questions 

Typical Debriefing Chart No. 2 - Agenda 
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SOURCE SELECTION “BLUEPRINT” 

T3pical Unsuccessful Offeror Debriefing Charts - Post Award 


SOURCE SELECTION PROCESS 
Source Selection Team 


Source Selection Authority 



Proposal Eva 

1 

iluation Board 

1 


1 

Technical 

Cost 


Past Performance 


Typical Debriefing Chart No. 3 - Source Selection Process 


186 





SOURCE SELECnON “BLUEPRINT” 

Typic^ Unsuccessful Offeror Debriefing Charts - Post Award 


EVALUATION RATINGS DEFINITIONS 


EXCELLENT: The proposd demonstrates an excellent understanding of requirements and approach 
that sigdficantty ex ceeds performance or capability standards. It has ex ceptional strengths that will 
significantly benefit the Government. 

GOOD: The proposal demonstrates a goodmderstandingof the requirements and an ^proach that 
ex ceeds performance or capsbilily standards. It has one or more strengths that will benefit the 
Government. 

SATISFACTORY: The proposal demonstrates an acc^table understanding cf requirements and an 
approach that meets performance or capability standards. It demonstrates an acceptable sdutior; 
however, has few or no strengttis. 

MARGINAL: The proposal demonstrates didlow understanding of the requirements and an approach 
that only margnally meets performance or capability standards necessary for minimal but acceptable 
contract performance. 

UNSATISFACTORY: The proposal fails to meet performance and capability standards. The 
requirements can only be met withmaj or changes to the prcposal. 

Typical Debriefing Chart No. 4 - Evaluation Ratings 
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SOURCE SELECTION “BLUEPBINT” 

Tjppical Unsuccessful Offeror Debriefing Charts - Post Award 


FACTOR EVALUATION RESULTS 


Unsuccessful Successful 

Corp. Corp. 


Technical Merit 

Good 

Excellent 

significantly more important than 



Proposal Risk 

Moderate 

Low 

which is equal to 



Past Performance Risk 

Low Risk 

Low Risk 

which is equal to 



Cost 

$1,095,200 

$1,186,100 


Typical Debriefing Chart No. 5 - Evaluation Results 
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SOURCE SELECTION “BLUEPRINT” 

Typicd Unsuccessful Offeror Debriefing Charts - Post Award 

STRENGTHS/WEAKNESSES 
Unsuccessful Corp. 

SIGNinCANT STRENGTHS 

• Outstanding hardware architecture. 

• Excellent cross utilization of personnel. 

• Extensive corporate experience in high performance computer hardware. 

• Outstanding internal control and quality assurance procedures. 

• Use of verification requirements traceability matrix. 

• Excellent understanding of data requirements. 

• Excellent understanding of unique platform requirements. 

SIGNinCANT WEAKNESSES 

• Software is not user fnendly. 

• Lack of corporate experience in software development. 

♦Insufficient detail on possibility of concurrent installation/training. 

• Part-time commitment of some key personnel to other projects. 

Typical Debriefing Chart No. 6 - Strenglhs/Weaknesses 
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SOURCE SELECTION“BLUEPEINT’’ 

Typical Unsuccessful Offero r Debriefing Charts - Post Award 

TECHNICAL MERIT SUBFACTORS 


Unsuccessful Successful 

Corp. Corp. 


Technical Merit 

Good 

Excellent 




Hardware Design 

Bccellent 

Excellent 



Software Configuration 

Satisfactory 

Excellent 



Corporate Experience 

Excellent 

Excellent 

rrrtr' a T-kT? T 


ALL ELEMENTS ARE EQUAL 


Typical Debriefing Chart No. 7 - Technical Merit 


190 









SOURCE SELECTION“ELUEPHINT” 

Typical Unsuccessful Offeror Debriefing Charts - Post Award 


PROPOSAL RISK SUBFACTORS 


Unsuccessful Successful 

Corp. Corp. 



ALL ELEMENTS ARE EQUAL 


Typical Debriefing Chart No. 8 - Proposal Risk 
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SOURCE SELECnON “BLUEPRINT” 

Typical Unsuccessful Offeror DdbrieGng Charts - Post Award 


PAST PERFORMANCE SUBFACTORS 


L 

Unsuccessful Successful 

Corp. Corp. 

Performance Risk 

Low Risk 

Low Risk 




Quality of Performance 

Low Risk 

Low Risk 



Cost Management 

Moderate Risk 

Low Risk 



Contract Schedules 

Low Risk 

Moderate Risk 



Customer Relations 

Low Risk 

ALL 

Low Risk 

ELEMENTS ARE EQUAL 


Typical Debriefing Chart No. 9 - Performance Subfactors 
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Typical DebrieEing Chart No. 10 - Cost 
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SOURCE SELECTION “BLUEPRINT” 

Typical Unsuccessful Offeror Debriefing Charts - Post Award 

COST REALISM ADJUSTMENTS 
Unsuccessful Corp. 


Proposed Adj. Evaluated 

G&A Expense $96,678 + $8,320 $104,998 
Travel $ 4,600 +$12,680 $17,280 

Typical Debriefing Chart No. 11 - Cost Realism Adjustments 
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SOURCE SELECTION“BLTJEPEINT” 

T}pical Unsaccessful Offeror Debriefing Charts - Post Award 


RATIONALE FOR AWARD DECISION 


Successful Corp’s innovative approach to software 
configuration represents a significant advancement in the state 
of the art in terms of capability and user fiiendly interface. Its 
proposed hardware architecture is superior to the other offerors 
in terms of processing speed. Successful Corp. has extensive 
experience in the development of high performance computer 
systems and its past performance on similar contracts was 
excellent Overall, Successfiil Corp’s proposed solution 
coupled with its low risk represents the best value to the 
Government despite its slightly higher cost 

Typical Debriefing Chart No. 12 - Aware Decision Rationale 





SOURCE SELECTION “BLUEPEINT” 

Typical Unsuccessful Offeror Debriefing Charts - Post Award 


OVERALL RANKING OF PROPOSALS 


1. Successful Corp. 

2. Offeror C 

3. Unsuccessful Corp. 

4. Offeror E 

5. Offeror B 

6. Offeror A 


Typical Debriefing Chart No. 13 - Proposal Rankings 

Blueprint: Typical Unsuccessful Offeror Debriefing Charts — Post Award 

Whos in Charge? 

The contracting officer is responsible for the debriefing and as such must never lose control of 
the debriefing. 


Introduction 

The debriefing should begin with a brief introduction of all attendees. You should then explain 
the purposes of the debriefing. (Refer to: What is a debriefing?) 

Ground Rules 

Next, you should inform everyone of the ground rules for the debriefing, any time constraints, 
and the debriefing agenda. 


Handling Questions 

During both preaward and postaward debriefings, the debriefed offeror is permitted to ask 
relevant questions pertaining to whether the Government followed the source selection 
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procedures set forth in the solicitation, applicable regulations, and other applicable authorities. 
Often, contracting officers request that questions from the debriefed offeror’s personnel be 
funneled to the debriefed offeror’s main spokesperson who will ask all the questions. This 
practice is permissible and, in fact, often enhances the orderly conduct of a debriefing. 

The contracting officer must make every effort to provide reasonable responses to those 
questions. However, caution must be exercised not to inadvertently disclose proprietary 
information of other offerors. 

Normally, responses should stay within the confines of the guidance stated above. If the 
contracting officer is unprepared to answer a specific question at the debriefing, the contracting 
officer should obtain the answer immediately following the debriefing and promptly furnish it to 
the offeror. To avoid this, you may find it helpful to request written questions ahead of time to 
use in preparing for the debriefing. 

Because the debriefing is the forum for answering reasonable questions, offerors should not be 
invited to submit questions after the debriefing. 

Source Selection Process 

The process used in evaluating proposals, establishing the competitive range, and selecting the 
awardee should be briefly explained. When conducting a postaward debriefing, the source 
selection authority normally should be identified by name. The identities of the evaluators, other 
than those present at the debriefing, are normally not disclosed. Nonetheless, the Government 
must convey to the offeror that the proposals were evaluated by qualified personnel. 

Evaluation Factors/Subfactors 

The evaluation factors and subfactors disclosed in the solicitation should be restated as the 
foundation for discussing the evaluation results. 

Evaluation Results 

The source selection information disclosed at debriefings for an unsuccessful offeror is governed 
by law, regulation and the sound discretion of the contracting officer. If the contracting officer 
observes the following guidelines for pre and postaward debriefings, there should be little 
concern about inadvertently disclosing unauthorized proprietary information. 

Preaward Debriefings 

At a Tninirnum, the following information will be disclosed: 


(1) The evaluation results of significant 
elements in the debriefed offeror’s proposal. 
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What are significant elements? If the element was significant enough to eliminate the offeror 
fi^om the competitive range, it is probably significant for debriefing purposes. Include in your 
discussion positive elements of the offeror’s proposal to help him improve future proposals. 


(2) Summary of the rationale for eliminating 
the offeror from the competition. 


What Cannot Be Disclosed! (Preaward) 

In addition to the information that may Not be disclosed in postaward debriefings 
(see What Cannot be Disclosed! (Postaward)), by law the following information 
may not be disclosed in a preaward debriefing: 

(1) The number of offerors; 

(2) The identity of other offerors; 

(3) The content of other offeror*s proposals; 

(4) The ranking of other offerors; 

(5) The evaluation of other offerors. 

Postaward Debriefings 

At a minimum, the following information will be disclosed: 


(1) The deficiencies and significant weaknesses 
of the debriefed offeror’s proposal. 


When is a weakness significant? If the weakness was of significant enough concern to warrant 
its discussion during the negotiation phase of the acquisition, it is probably significant for 
debriefing purposes as well. Whereas, if it wasn’t sigmficant enough to warrant discussion, it is 
not significant for debriefing purposes either, unless, of course the weakness was created in the 
final proposal revision. It is also a good practice to discuss the sigmficant advantages of the 
debriefing offeror’s proposal. 


(2) The evaluation ratings of the 
debriefed offeror and awardee — but only 
to the second level of evaluation. 
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If the evaluation board used adjectival ratings, the adjectives and their definitions contained in 
the evaluation plan should be disclosed. If numerical ratings or color codings were used instead, 
they should be disclosed likewise. 

What is the second level of evaluation? For example, assume the solicitation sets forth the 
following four evaluation factors: Technical, Management, Integrated Logistics 

Support (ILS), and Past Performance. This is the first level of evaluation and the overall ratings 
for each of the four factors would be disclosed. 

If several subfactors were separately rated under a factor (such as Management Approach, 
Proposed Staffing, and Past Corporate Experience in the Management factor), then those 
subfactors constitute the second level of evaluation and their ratings should also be released. 

The evaluation ratings at the third and fourth levels of evaluation of the awardee’s proposal 
should normally not be revealed. If ratings are revealed at these lower levels, it is conceivable 
that the contracting officer may nm afoul of the statutory prohibition against point-by-point 
comparisons of proposals. 

Ratings of the debriefed offeror’s proposal may be revealed to these lower levels if necessary to 
explain the rationale for the award decision. Be prepared to explain the rationale for the ratings 
of the debriefed offeror’s proposal. 


(3) The government’s total 
evaluated costs/prices of the 
debriefed offeror’s proposal. 


The total evaluated costs/prices of the debriefed offeror’s proposal should be disclosed for each 
contract line item (CLIN), and an explanation given for any significant cost realism adjustments 
made by the Government at the major cost element level. 


(4) The total evaluated costs/prices 
of the awardee’s proposal. 


The awardee’s total proposed and evaluated costs/prices for each CLIN should be disclosed. 

Do Not disclose the specific Government cost/price adjustments to the awardee’s proposed 
cost/prices. 


(5) Overall ranking of all proposals. 
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If the source selection authority ranked the proposals (i.e., the best overall proposal, 2d best, 
etc.), the overall ranking of all proposals must be revealed. However, the identities of the other 
unsuccessful offerors should not be revealed. Refer to those offerors by alphanumerical letters 
or other designators. 


(6) Rationale for award decision. 


The Government must disclose a summary of the rationale for the contract award decision. The 
rationale is contained in the source selection authority’s (SSA) decision memorandmn. 
Contracting officers should fiimish the debriefed offeror a copy of the SSA’s memorandum, if 
requested. However, evaluation information concerning the other unsuccessful offerors and 
proprietary information of the award must be redacted prior to release. 

Identify the significant advantages of the awardee’s proposal in general terms without revealing 
confidential proprietary information contained in the awardee’s proposal. (See below, What 
Cannot Be Disclosed!) 


(7) The make and model of any commercial 
end items proposed by the awardee. 


If the awardee’s proposal includes a commercial item that is an end item imder the contract, the 
make and model of Ae item must be disclosed. 


Information Pertaining To Other 
Unsuccessful Offerors That May 
Be Released (Postaward) 


Under certain circumstances, you may release other information such as the final 
overall ratings for noncost factors and/or the final evaluated cost/price of the other 
unsuccessful offerors. Release of the overall noncost ratings is discretionary. However, 
release of the total final evaluated cost/price is limited to those situations where an 
unsuccessful offeror consents or the agency determines that the unsuccessful offeror, 
after consulting with it, would not suffer competitive harm from such a release. The 
decision to release any of this information must be made on a case-by-case basis with 
guidance from your legal office. 

What Cannot Be Disclosed! (Postaward) 

By law a debriefing may Not include point-by-point comparisons of the debriefed offeror’s 
proposal with the other proposals. If the ratings of the proposals of the debriefed offeror and the 
awardee are disclosed only to the second level of evaluation, this prohibition should not present a 
problem. 
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Also by law, debriefings may NOT disclose information that is exempt from release under the 
Freedom of Information Act (FOIA) relating to: 

(1) Trade secrets; 

(2) Privileged or confidential manufacturing processes and techniques; 

(3) Commercial and financial information that is privileged or confidential, including 
cost breakdowns, profits, indirect cost/rates, and similar information; and 

(4) Names of individuals providing reference information about an offeror’s past 
performance. 

This information is normally referred to as “proprietary information. ” 

Proprietary information means information contained in a proposal or 
otherwise submitted to the Government that the submitter has marked 
as proprietary. Proprietary information does Not include information 
that is otherwise available without restriction to the Government or the 
public. If you believe that information marked proprietary is not truly 
proprietary, you should contact the assigned legal advisor for an 
appropriate determination before the information is released. 

Debriefing the Awardee 

Although debriefing an awardee is similar in many respects to debriefing an unsuccessful 
offeror, there is one significant difference - very little information is revealed regarding the 
proposals of the unsuccessful offerors. 

As discussed in Debriefing The Unsuccessful Offeror, the following outline is suggested: 

• Introduction. 

• Explain the purposes of the debriefing. 

• Announce the ground rules. 

• Summarize the source selection process that was used. 

• State the proposal evaluation factors and subfactors. 

• Reveal the evaluation results: 

(1) The significant advantages of the awardee’s proposal. 

(2) The significant weaknesses of the awardee’s proposal. 

(3) The evaluation ratings of the awardee’s proposal to the second level of 
evaluation. Explain the rating definitions. 

(4) The Government’s total evaluated costs/prices of the awardee’s proposal for 
each Contract Line Item (CLIN). Explain significant cost realism adjustments 
made by the Government to the major cost element level. 
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A summary of the rationale for the contract award decision. 

The overall ranking of all proposals (if overall rankings were made during source 
selection), but do not identify the unsuccessful offerors by name. 

Ans wer relevant questions pertaining to whether the Government followed the 
source selection procedxnes set forth in the solicitation, applicable regulations, and 
other authorities. 


The Postdebriefing Memorandum 

The contracting officer must include a summary of each debriefing in the contract file. Good 
postdebriefing memorandums are essential if the acquisition is reopened or resolicited as a result 
of a protest or otherwise within 1 year of the contract award date. 

In those circumstances, the law requires that the contracting agency make available to all 
offerors information regarding the proposal of the awardee that was provided to other offerors at 
debriefings on the prior contract. This requirement is designed, in part, to place all offerors on a 
level playing field. Thus, the need for good postdebriefing memorandums is apparent. 

The postdebriefing memorandum should include at a mimmum: 

• A list of all persons who attended the debriefing. 

• A summary of the information disclosed during the debriefing. The most efficient 
means for doing this is to identify the charts that were used at the debriefing and 
attach a copy of them to the memorandum. 

• The substance of all questions and answers discussed at the debriefing. Include 
answers provided after the debriefing. 
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APPENDIX C. 

PROTESTS OF CONTRACTS AWARDED USING A TRADE-OFF PROCESS 


GAO Case 


Number 

Case Name 

Decision Date 

Agency 

Decision 

283057 

Ocean House Builders 

21-Sep-99 

Air Force 

Denied 

278732 

Strategic Resource Solutions Corp. 

09-Mar-98 

Air Force 

Denied 

280758 

Avtron Manufacturing, Inc. 

16-NOV-98 

Air Force 

Denied 

281200 

CDS Network Systems, Inc. 

21-Dec.98 

Air Force 

Denied 

281046 

PCT Services, Inc. 

22-Dec-98 

Air Force 

Denied 

281222 

WRS Infi^tructure & Environment, Inc. 

12-Jan-99 

Air Force 

Denied 

281275 

QSS Information Systems, Inc. 

19-Jan-99 

Air Force 

Denied 

281350 

Saratoga Medical Center, Inc. 

27-Jan-99 

Air Force 

Denied 

280521.3 

RMS Information Systems, Inc. 

21-Oct-98 

Air Force 

Denied 

283011 

Engineered Air Systems, Inc. 

21-Sep-99 

Air Force 

Denied 

280521.2 

Techniacts Science & Technology Corp. 

15-Oct-98 

Air Force 

Denied 

283028 

Premier Engineering & Manufacturing 

Inc. 

21-Sep-99 

Air Force 

Denied 

283080 

Acepex Management Corp. 

04-Oct-99 

Air Force 

Denied 

283272 

Alaskan Publications 

27-Oct-99 

Air Force 

Denied 

278591 

Beneco Enterprises, Inc. 

Ol-Feb-98 

Air Force 

Denied 

278582.2 

CPAD Technologies, Inc. 

19-Feb-98 

Air Force 

Denied 

283794 

Tony’s Landscaping, Inc. 

23-NOV-99 

Air Force 

Denied 

278695 

Modem Technologies Corp., et.al. 

04-Mar-98 

Air Force 

Denied 

282272 

Inland Service Corp. 

21-Jun-99 

Air Force 

Denied 

279839 

Rothe Development, Inc. 

27-Jul-98 

Air Force 

Denied 

281359 

West Coast Research Coip. 

Ol-Feb-99 

Air Force 

Denied 
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GAO Case 
Number 

Case Name 

Decision Date 

Agency 

Decision 

281388 

OMV Medical, Inc., et.al. 

03-Feb-99 

Air Force 

Denied 

281387 

OMV Medical, Inc., etal. 

03-Feb-99 

Air Force 

Denied 

281453 

Orbital Technologies Corp. 

17-Feb-99 

Air Force 

Denied 

281965 

ENMAX Coip. 

12“May-99 

Air Force 

Sustained 

280996.2 

NAPA Supply of Grand Forks, Inc. 

13-May-99 

Air Force 

Denied 

282064 

Si-Nor, Inc. 

25-May-99 

Air Force 

Sustained 

280154.2 

Rohmann Services, Inc. 

16-NOV-98 

Air Force 

Denied 

279173.5 

Acepex Managenient Corp. 

22-Jul-98 

Air Force 

Denied 

282373 

Spectrum Sciences & Software, Inc. 

22-Jun-99 

Air Force 

Denied 

279892 

Cache Box, Inc. 

29-Juh98 

Air Force 

Denied 

278855.2 

Apache Enterprises, Inc. 

30-JuI-98 

Air Force 

Denied 

279191.3 

Main Building Maintenance, Inc. 

05-Aug-98 

Air Force 

Denied 

2784246.1 

Symetrics Industries, Inc. 

17-Sep-98 

Air Force 

Denied 

278478.4 

CVB Company 

21-Sep-98 

Air Force 

Denied 

280374 

B3H Corp. 

23-Sep-98 

Air Force 

Denied 

280521 

Bionetics Corp. 

14-Oct-98 

Air Force 

Denied 

282082 

Buckeye Park Services, Inc. 

Ol-Jun-99 

Air Force 

Denied 

279251.2 

Exploration Products 

Ol-Jun-98 

Air Force 

Denied 

278897.4 

WP Photographic Services 

12-May-98 

Air Force 

Denied 

278165 

International Consultants, Inc.,etal. 

05-Jan-98 

Air Force 

Denied 

279168 

PCT Services, Inc. 

12-May-98 

Air Force 

Denied 

275587.9 

Pemco Aeroplex, Inc., Aero Corp. 

29-Jun-98 

Air Force 

Denied 

279811 

Applied Companies 

24-Jul-98 

Army 

Denied 

280362 

Caterpillar, Inc. 

23-Sep-98 

Army 

Denied 

278408 

Thermolten Tech., Inc. 

26-Jan-98 

Army 

Denied 
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278408.3 

Teledyne-Commodore, LLC 

15-Sep-98 

Army 

Dismissed 

281136 

AMS Mechanical Systems, Inc. 

04-Jan-99 

Army 

Denied 

278129.4 

Proteccion Total/Magnum Security, S.A. 

12-May-98 

Army 

Denied 

278651 

Compro Computer Services, Inc. 

23-Feb-98 

Army 

Denied 

280133 

Electronic Data Systems Corp. 

03-Sep-98 

Army 

Denied 

283307 

ITT Federal Services International Corp. 

03-NOV-99 

Army 

Sustained 

278358 

Compania De Asesoria Y Comercio, S.A. 

20-Jan-98 

Army 

Denied 

283393 

Charleston Marine Containers, Inc. 

08-NOV-99 

Army 

Denied 

280520 

Buck Environmental Technologies, LLC 

14-Oct-98 

Army 

Denied 

278195.2 

Robotic Systems Technology 

07-Jan-98 

Army 

Denied 

278673 

Battelle Memorial Institute 

27-Feb-98 

Army 

Denied 

283259 

Govt, of Hartford County, Maryland 

28-Oct-99 

Army 

Denied 

279561 

Trifax Corp. 

29-Jun-98 

Army 

Sustained 

280422.3 

RTF/TCI/EAI Joint Venture 

29-Dec-98 

Army 

Denied 

282334 

Canadian Commercial Corp./Liftking 
Industries, Inc. 

30-Jun-99 

Army 

Denied 

279723 

Riveer Company 

14-Jul-98 

Army 

Denied 

282162 

TRW, Inc. 

09-Jun-99 

Army 

Denied 

280978.3 

The Arora Group, Inc. 

27-Jan.99 

Army 

Denied 

281342 

Lance Ordnance, Inc. 

26-Jan-99 

Army 

Denied 

282511 

ATA Defense Industries, Inc. 

21-Jul-99 

Army 

Sustained 

278298 

Midmark Corp. 

14-Jan-98 

Army 

Denied 

281246 

Vision Blocks, Inc. 

14-Jan-99 

Army 

Denied 

283401 

Olin Corp. 

15-NOV-99 

Army 

Denied 

281006 

RMG Systems, Ltd. 

18-Dec-98 

Army 

Denied 
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280988 

NWT, Inc., PharmChem Laboratories, 

Inc. 

17-Dec-98 

Army 

Denied 

280585 

Input/Output Technology, Inc. 

21-Oct-98 

Army 

Denied 

280568 

Day & Zimmerman/IMR, LLC 

19-Oct-98 

Army 

Denied 

279393 

Marlen C. Robb & Son Boatyard & 

Marina Inc. 

09-Jun-98 

Army 

Denied 

278324.2 

Noelke GmbH 

09-Feb-98 

Army 

Denied 

278319 

Electro-Voice, Inc. 

15-Jan-98 

Army 

Denied 

281289 

APTUS Company 

20-Jan-99 

Army 

Denied 

278168 

Joint Threat Services 

05-Jan-98 

Army 

Denied 

281331.3 

Kellie W. Tipton Construction 

Company 

22-Mar-99 

Army 

Denied 

278797 

Rockhill Industries, Inc. 

16-Mar-98 

Army 

Denied 

278408.5 

Teledyne-Commodore, L.L.C. 

08-Mar-99 

Army 

Denied 

278689 

Jason Associates, Corp. 

02-Mar-98 

Army 

Denied 

279561.2 

Professional Performance Development 
Group, Inc. 

06-Jul-98 

Army 

Denied 

278730 

FC Business Systems, Inc. 

06-Mar-98 

Army 

Sustained 

281141.3 

Clean Service Company, Inc. 

16-Feb-99 

Army 

Denied 

277827.5 

Marquette Medical Systems, Inc. 

29-Apr-99 

Army 

Sustained 

280766.3 

E.L. Hamm & Associates, Inc. 

12-Apr-99 

Army 

Sustained 

279637.2 

Hugo-Cantu Joint Venture 

20-Jul-98 

Army 

Denied 

283827 

B. Diaz Sanitation Inc. 

27-Dec-99 

Army 

Denied 

277241.2 

Aalco Forwarding, Inc., etal. 

Ol-Jul-98 

Army 

Denied 

279585 

IT Facilities Services 

29-Jun-98 

Army 

Denied 

281384 

Bulora Technologies, L.L.C. 

03-Feb-99 

Army 

Denied 

281002.2 

Andrulis Corp. 

02-Jun-99 

Army 

Denied 

278793 

Coleman Research Corp. 

16-Mar-98 

Army 

Denied 
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277241.15 

Aalco Forwarding, Inc., et al. 

ll-Mar-98 

Army 

Denied 

281244.3 

D & K Construction Company, Inc. 

Ol-Oct-99 

Army 

Denied 

282739 

Allied Technology Group, Inc. 

19-Aug-99 

Army 

Denied 

278371 

Hardcore Dupont Composites, L.L.C. 

20-Jan-98 

Army 

Denied 

282140 

North State Resources, Inc. 

07-Jun-99 

Army 

Denied 

280044.2 

HAP Construction, Inc. 

21-Sep-98 

Army 

Denied 

279492.2 

Black & Veatch Special Projects Corp. 

26-Jun-98 

Army 

Denied 

282231 

Garco Construction, Inc., etal. 

15-Jun-99 

Army 

Denied 

281082 

Omni Corp. 

22-Dec-98 

Army 

Dismissed 

281695 

Stratus Systems, Inc. 

24-Feb-99 

DoT 

Denied 

283882 

Phantom Products, Inc. 

30-Dec-99 

DoT 

Denied 

278422 

Baldt, Inc. 

28-Jan-98 

DoT 

Denied 

279217 

Quaker Valley Meats, Inc./Supreme Sales, 
Upper Lakes Foods 

20-May-98 

DCA 

Denied 

280969 

MCR Federal, Inc. 

14-Dec-98 

DFAS 

Sustained in part 

280463.3 

Rel-Tek Systems & Design, Inc, 

25-NOV-98 

DFAS 

Sustained 

281393 

Deva & Associates, PC. 

01-Feb-99 

DFAS 

Denied 

283491 

Communications Technologies, Inc. 

30-NOV-99 

DISA 

Denied 

281876 

Marathon Watch Company Ltd. 

22-Apr-99 

DLA 

Denied 

282554 

Wirt Inflatable Specialists, Inc. 

28-Jul-99 

DLA 

Denied 

280180.2 

The Terxe Group, Inc. 

28-Sep-98 

DLA 

Denied 

278268.2 

MCA Research Corp. 

lO-Apr-98 

DLA 

Denied 

278881 

San Diego Beverage & Cup, Inc. 

24-Mar-98 

DLA 

Denied 

280872.2 

Venture Productions 

09-Dec-98 

DLA 

Denied 

282150 

Wellco Enterprises, Inc. 

04-Jun-99 

DLA 

Denied 

281958 

National Aerospace Group, Inc. 

lO-May-99 

DLA 

Sustained in part 
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280767.4 

D.F. Zee's Fire Fighter Catering 

lO-Sep-99 

DoA 

Denied 

2S07672 

Wilderness Mountain Catering 

28-Dec-98 

DoA 

Denied 

280218 

Sawtooth Enterprises, Inc. 

07-Dec-98 

DoA 

Denied 

279272 

William C. Menefee 

28-May-98 

DoA 

Denied 

281497 

Crasehi Rentals, Inc. 

17-Feb-99 

DoA 

Denied 

277063.3 

Tri-J Contractors 

06-Jul-98 

DoA 

Denied 

281070.2 

West Coast Unlimited 

18-Aug-99 

DoA 

Denied 

283177 

GSA, Inc. 

18-Oct-99 

DoA 

Denied 

278579.2 

Possehn Consulting 

29-Jul-98 

DoA 

Sustained 

280774.2 

R.C.O. Reforesting 

24-N0V-98 

DoA 

Denied 

278659 

Seedburo Equipment Co. 

25-Feb-98 

DoA 

Denied 

281706 

LB&B Associates, Inc. 

24-Mar-99 

DoA 

Denied 

277572.4 

HG Properties A., L.P. 

ll-Feb-98 

DoA 

Denied 

278278 

Biospherics, Inc. 

14-Jan-98 

DoA 

Sustained 

278649 

High County Contracting 

30-Jan-98 

DoA 

Denied 

278579 

Possehn Consulting 

09-Jan-98 

DoA 

Sustained 

283137.3 

Kathpal Technologies, Inc., et.al. 

30-Dec-99 

DoC 

Sustained 

280003 

Madico, Inc. 

12-Aug-98 

DoD (HQ) 

Denied 

278904.4 

Centech Group, Inc. 

13-Apr-98 

DoE 

Denied 

278904.3 

Techsys Corp. 

13-Apr-98 

DoE 

Denied 

278904.2 

McNeil Technologies, Inc. 

02-Apr-98 

DoE 

Denied 

277801.4 

Advanced Data Concepts, Inc. 

Ol-Jun-98 

DoE 

Denied 

278044.5 

L & M Technologies, Inc. 

08-May-98 

DoE 

Denied 

282326 

Johnson Controls, Inc. 

28-Jun-99 

DoE 

Denied 
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280967.8 

Advanced Data Concepts, Inc. 

14-Jun-99 

DoE 

Denied 

278451 

COMPA Industries, Inc. 

30-Jan-98 

DoE 

Denied 

278904 

KRACorp. 

02-Apr-98 

DoE 

Denied 

277805.2 

Scientech, Inc. 

20-Jan-98 

DoE 

Sustained 

278869 

MicroMass, Inc. 

24-Mar-98 

DoE 

Denied 

278044.4 

Source One Management, Inc. 

12-Jun-98 

DoE 

Denied 

280929 

Westinghouse Government & 

Environmental Services Co, Inc. 

04.Dec-98 

DoE/Navy (joint) 

Denied 

280447 

YKK(USA),Inc. 

28-Aug-98 

DoJ 

Denied 

281074.2 

National Toxicology Laboratories, Inc. 

11-Jan-99 

DoJ 

Denied 

279048.2 

Professional Gunsmithing, Inc. 

24-Aug-98 

DoJ 

Denied 

282139 

Interlog, Inc. 

27-Apr-99 

DoJ 

Denied 

280388 

General Security Services Corp. 

25-Sep-98 

DoJ 

Denied 

281484.2 

Medical Development International 

29-Mar-99 

DoJ 

Denied 

280595 

Keane Federal Systems, Inc. 

23-Oct-98 

DoJ 

Denied 

280959 

General Security Services, Corp. 

ll-Dec-98 

DoJ 

Denied 

281051.5 

Warvel Productions, Inc. 

07-Jul-99 

DoJ 

Etenied 

276633.2 

RAMCOR Services Group, Inc. 

23-Mar.98 

DoJ 

Denied 

278906 

M3 Corp. 

01-Apr-98 

DoT 

Denied 

278276.11 

Patriot Contract Services, LLC, et.al. 

22-Sep-98 

DoT 

Denied 

278466 

Hughes STX Corp. 

02-Feb-98 

DoT 

Sustained 

280270 

Randolph Engineering Sunglasses 

lO-Aug-98 

DLA 

Denied 

280645 

Phillips Industries, Inc. 

17-Sep-98 

DLA 

Denied 

281517 

Island Components Group, Inc. 

19-Feb-99 

DLA 

Denied 

279697 

Dynamic Marketing Services, Inc. 

13-Jul-98 

Education 

Denied 
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277029.3 

McAllister & Associates, Inc. 

18-Feb-98 

Education 

Denied 

279924 

Information Ventures, Inc. 

07-Aug-98 

EPA 

Denied 

279379.2 

TEAM Support Services, Inc. 

22-Jun-98 

EPA 

Denied 

280011.2 

Nick Chorak Mowing 

01 .Oct-98 

EPA 

Denied 

278650 

USA Today 

20-Feb-98 

DoJ 

Denied 

281836 

Construction Technology Laboratories, 

Inc. 

12-Apr-99 

DoT 

Denied 

283538 

Simborg Development, Inc. 

07-Dec-99 

GSA 

Denied 

279715 

McHargue Construction Co. 

16-Jul-98 

GSA 

Denied 

281995 

Concepts Building Systems, Inc. 

13-May-99 

GSA 

Denied 

280518 

Acquest Uniland LLC 

13.0ct-98 

GSA 

Denied 

283398 

Ideal Electronic Security Company, Inc. 

lO-Nov-99 

GSA 

Denied 

283192 

S.J. Thomas Co., Inc. 

2043ct-99 

GSA 

Sustained 

277874.4 

Correa Enterprises, Inc. 

13-May-98 

GSA 

Denied 

276837.3 

Omega World Travel, Inc. 

09-Jun-98 

GSA 

Denied 

279295 

Dual, Inc. 

Ol-Jun-98 

GSA 

Denied 

278876.2 

NAAB Research Center, Inc. 

04-May-98 

GSA 

Denied 

278851 

McShade Enterprises 

23-Mar-98 

GSA 

Denied 

282776 

U.S. Constructors, Inc. 

21-Jul-99 

GSA 

Denied 

276012.2 

Wackenhut Services, Inc. 

Ol-Sep-98 

GSA 

Denied 

280475 

Cobra Technologies, Inc, 

06-Oct-98 

GSA 

Denied 

282456 

Universal Building Maintenance, Inc. 

15-Jul-99 

GSA 

Sustained 

280013.2 

Information Systems Technology Corp. 

06-Aug-98 

HHS 

Denied 

280176 

California Resources 

08-Sep-98 

HHS 

Denied 

278508.4 

Biospherics, Inc. 

06-Oct-98 

HHS 

Sustained 
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279260 

Conwal, Inc. 

26-May-98 

HHS 

Denied 

282074 

GCI Information Services, Inc. 

28-May-99 

HHS 

Denied 

278343 

Comark Federal Systems 

20-Jan-98 

HHS 

Sustained 

278678 

The American Indian Center, Inc. 

27-Feb-98 

HHS 

Denied 

278839 

I.S. Grupe, Inc. 

20-Mar-98 

HHS 

Denied 

279378 

Century Environmental Hygiene, Inc. 

05-Jun-98 

HHS 

Sustained 

274689.5 

/ 

Tidewater Homes Realty, Inc. 

ll-Aug-98 

HUD 

Denied 

279161.2 

Ervin & Associates 

13-Oct-98 

HUD 

Denied 

282098 

ACS Government Solutions Group, Inc. 

02-Jun-99 

HUD 

Sustained 

281352.14 

Housing Services, Inc. 

07-May-99 

HUD 

Denied 

278727 

U.S. Property Management Service 

COTp. 

06-Mar-98 

HUD 

Sustained 

283147 

The Communities Group 

I2.0ct-99 

HUD 

Denied 

279083 

Ervin & Associates, Inc. 

30-Apr-98 

HUD 

Etenied 

280993 

Ervin & Associates, Inc. 

17-Dec-98 

HUD 

Denied 

280283.3 

Du & Associates, Inc. 

22-Dec-98 

HUD 

Denied 

279565.2 

Consolidated Engineering Services, Inc. 

26-Jun-98 

HUD 

Sustained 

279565.5 

Consolidated Engineering Services, Inc. 

19-Mar-99 

HUD 

Denied 

282808 

Cotton & Company, LLP 

30-Aug“99 

HUD 

Sustained 

281745.2 

Federal Security Systems, Inc. 

29-Apr-99 

Interior 

Denied 

280478 

Emmert International 

07-Oct-98 

Interior 

Denied 

282192 

Matrix General, Inc. 

lO-Jun-99 

Interior 

Denied 

283469.2 

Crescent Helicopters 

30-NOV-99 

Interior 

Denied 

282661.2 

East Slope Conservation Services 

09-Jul-99 

Interior 

Denied 

278450 

American Artisan Productions, Inc. 

30-Jan-98 

Interior 

Denied 
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283822 

Century Elevator Inc. 

20-Dec-99 

JFK-CPA 

Denied 

277849.2 

Sytel, Inc. 

08-Jan-98 

NASA 

Denied 

278896.2 

Aerospace Design & Fabrication 

04-May-98 

NASA 

Sustained 

283160 

Scientific & Commercial Systems Corp., 
et.al. 

14-Oct-99 

NASA 

Denied 

281773.2 

Mentor Technologies, Inc.,et.al. 

01-Apr-99 

NASA 

Denied 

281773 

SGT, Inc. 

Ol-Apr-99 

NASA 

Denied 

280664.2 

Plasma-Therm, Inc. 

28-Dec-98 

NASA 

Denied 

279828 

Technical & Administrative Services 

Corp. 

24-Jul-98 

NASA 

Denied 

279014 

P & R Water Taxi, Ltd. 

23-Apr-98 

Navy 

Dismissed 

278516.2 

Veda Inc. 

19-Mar-98 

Navy 

Denied 

279919 

SDV Telecommunications 

29-Jul-98 

Navy 

Denied 

281579 

Joppa Maintenance Company, Inc. 

02-Mar-99 

Navy 

Denied 

278260.2 

Davies Rail & Mechanical Works, Inc. 

25-Feb-98 

Navy 

Denied 

279543 

Hard Bodies, Inc. 

23-Jun-98 

Navy 

Denied 

281490 

OMV Medical, Inc. 

16-Feb-99 

Navy 

Denied 

279759.2 

Systems Integration & Research, Inc., 
etal. 

16-Feb-99 

Navy 

Denied 

280250 

Chant Engineering Co., Inc. 

07-Aug-9S 

Navy 

Denied 

277538.2 

Command Technology, Inc. 

28-Apr-98 

Navy 

Denied 

279707 

HSQ Technology 

09-Jul-98 

Navy 

Denied 

281521 

Chant Engineering Company, Inc. 

22-Feb-99 

Navy 

Denied 

278187 

J & E Associates, Inc. 

05-Jan-98 

Navy 

Denied 

280048.4 

Pulau Electronics Corp. 

19-May-99 

Navy 

Sustained in part 

279493.3 

Telos Corp. 

27-Jul-98 

Navy 

Denied 

283796 

B.E. Meyers & Company, Inc. 

14-Dec-99 

Navy 

Denied 
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278738 

Xeno Technix, Inc. 

11-Mar-98 

Navy 

Denied 

279793 

Pacific Ship Repair & Fabrication, Inc. 

23-Jul-98 

Navy 

Denied 

281872 

Metro Machine Corp. 

22-Apr-99 

Navy 

Sustained 

281784.3 

L-3 Communications Corp, Ocean 

Systems Division 

26-Apr-99 

Navy 

Sustained in part 

282437.3 

Bums & Roe Services, Corp, 

30-NOV-99 

Navy 

Dismissed 

279773 

Southwest Critical Care Associates 

16-Jul-98 

Navy 

Denied 

283650 

Advanced Communication Systems, Inc. 

16-Dec-99 

Navy 

Denied 

279662.5 

Electronic Design, Inc. 

25-May-99 

Navy 

Denied 

283738 

BioGenesis Pacific, Inc. 

14-Dec-99 

Navy 

Denied 

283448 

J & E Associates, Inc. 

03-NOV-99 

Navy 

Denied 

280814 

Packaging Strategies, Inc. 

25-NOV-98 

Navy 

Denied 

280511.2 

GTS Duratek, Inc. 

19-Oct-98 

Navy 

Sustained 

282982 

Thames Towboat Company, Inc. 

09-Sep-99 

Navy 

Denied 

281278 

Ventura Petroleum Services, Inc. 

21-Jan-99 

Navy 

Denied 

278141.2 

Dawco Construction, Inc. 

02-Jan-98 

Navy 

Denied 

281285 

Browning Ferris Industries of Hawaii, 

Inc. 

21-Jan-99 

Navy 

Denied 

280853 

Durocher Dock & Dredge/Black & 

Veatch, A Joint Venture 

24-NOV-98 

Navy 

Denied 

283236 

ELS, Inc. 

25-Oct-99 

Navy 

Denied 

280573.2 

Environmental Tectonics Corp. 

Ol-Dec-98 

Navy 

Denied 

280834 

Lear Siegler Services, Inc. 

25-NOV.98 

Navy 

Denied 

278281 

F2M-WSCI 

14-Jan-98 

Navy 

Denied 

282670 

A & T Engineering Technologies 

13-Aug-99 

Navy 

Denied 

282271 

Am Clyde Engineered Products 

Company, Inc. 

21-Jun-99 

Navy 

Denied 

277143.2 

Navistar Marine Instrument Corp. 

13-Feb-98 

Navy 

Denied 
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282887 

Olympus Building Services, Inc. 

31-Aug-99 

Navy 

Denied 

282912 

Infrared Technologies Corp. 

02-Sep-99 

Navy 

Denied 

283239 

DTH Management JV 

06-Oct-99 

Navy 

Denied 

282630.2 

OMNIPLEX World Services Corp. 

22-Sep-99 

Navy 

Denied 

282947 

Nova Group, Inc. 

15-Sep-99 

Navy 

Sustained 

283234 

J.A. Jones Grupo de Serviciois, SA. 

25-Oct-99 

Navy 

Denied 

277926.2 

Teleport Communications Group 

17-Sep-98 

Navy 

Denied 

280049 

Nations, Inc. 

24-Aug-98 

Navy 

Sustained 

278584 

U.S. Technology Corp. 

17-Feb-98 

Navy 

Denied 

279662,2 

Electonic Design, Inc. 

31~Aug-98 

Navy 

Sustained 

278734.4 

East/West Industries 

28-May-98 

Navy 

Denied 

281142 

National Steel & Shipbuilding Co, 

04-Jan-99 

Navy 

Denied 

280261 

Morrison Knudsen Corp. 

09-Sep.98 

Navy 

Denied 

281437.2 

Wilcox Industries, Corp. 

30-Jun-99 

Navy 

Sustained 

280284 

Aerostructures, Inc. 

15-Sep-98 

Navy 

Denied 

278647.2 

Oceanometrics, Inc. 

09-Jun-98 

Navy 

Denied 

282249 

J & D Maintenance & Service 

18-Jun-99 

Navy 

Denied 

282407 

Support Services, Inc. 

08-Jul-99 

Navy 

Denied 

278921.2 

Braswell Services Group, Inc. 

17-Jun-98 

Navy 

Denied 

282634 

Red River Service Corp. 

15-Jul-99 

Navy 

Denied 

278360 

Tidewater Construction Corp. 

20-Jan-98 

Navy 

Sustained 

280398.2 

Knights’ Piping Inc.,et.al. 

09-Oct-98 

Navy 

Denied 

281030 

Beckman Coulter 

21-Dec-98 

HHS 

Sustained 

279049 

Chant Engineering Company 

30-Apr-98 

PCC 

Denied 
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277916.2 

Mechanical Contractors, S.A. 

04-Mar-98 

PCC 

Sustained 

281503 

The Production Company 

18-Feb-99 

SS Admin. 

Denied 

279305 

WECO Cleaning Specialists, Inc. 

03-Jun-98 

SS Admin. 

Denied 

280429 

Securiguard/Group 4 Joint Venture 

30-Sep-98 

State 

Denied 

278929.7 

Jensco Marine, Inc. 

11-Feb-99 

Treasury 

Denied 

275502.3 

Signal Corp. 

06-Jul-98 

Treasury 

Denied 

281255 

Amdahl Corp. 

28-Dec-98 

Treasury 

Denied 

279221 

Ideal Electronic Security 

19-May-98 

Treasury 

Denied 

280922 

Teltara, Inc. 

04-Dec-98 

Treasury 

Sustained 

278929.2 

Mar, Inc., etal. 

28-Sep-98 

Treasury 

Denied 

281919 

KBM Group, Inc. 

03-May-99 

Treasury 

Denied 

278715 

The Centech Group, Inc. 

05-Mar-98 

Treasury 

Denied 

278189.3 

Foundation Health Federal Services, Inc. 
etal. 

04-Feb-98 

TSO 

Sustained 

278254 

Research Triangle Institute 

12-Jan-98 

USAID 

Denied 

279920 

Development Alternatives, Inc. 

06-Aug-98 

USAID 

Denied 

281241.2 

Ritchie Sawyer Corp. 

25-Jan-99 

USAID 

Denied 

281028.2 

Mid-Atlantic Service & Supply Corp. 

29-Dec-98 

USAID 

Denied 

280113 

RONCO Consulting Corp. 

11-Aug-98 

USAID 

Denied 

282555 

Chemonics International, Inc. 

23-Jul-99 

USAID 

Sustained 

282571 

AdvanChip Corp. 

29-Jul-99 

Interior 

Denied 

279756 

Robertson Leasing Corp. 

17-Jul-98 

DoJ 

Denied 

280959.5 

Famham Security, Inc. 

09-Feb-99 

DoJ 

Denied 

281904 

Walsh EHstribution, Inc. 

29-Apr-99 

VA 

Denied 
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GAO Case 


Number 

Case Name 

280430 

SDS Petroleum Products, Inc. 

2822243 

Life Oxygen & Health Services, Inc. 

281693 

Opti-Lite Optical 

278223 

BFI Waste Systems of Nebraska, Inc. 

278400 

University of Kansas Medical Center 

281693.2 

Opti-Lite Optical 

281800 

Korrect Optical 

280652 

HG Properties A., L.P. 

283295.2 

Rotech Medical Corp. 


Decision Date Agency Decision 


Ol-Sep-98 

VA 

Denied 

18-Jun-99 

VA 

Denied 

22-Mar-99 

VA 

Sustained 

08-Jan-98 

VA 

Denied 

26-Jan-98 

VA 

Denied 

15-Jul-99 

VA 

Denied 

09-Apr-99 

VA 

Denied 

02-NOV-98 

VA 

Denied 

08-NOV-99 

VA 

Denied 
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